Python/apps/py-cutils: cutils/treesum.py annotate

annotate cutils/treesum.py @ 347:3f4840eeb0f4

treesum: optimize help output: do not print "-h/--help" for "patterns" and "help"

author	Franz Glasner <fzglas.hg@dom66.de>
date	Tue, 01 Apr 2025 18:56:17 +0200
parents	d47965f97abb
children	539e2facbb28

rev	line source
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	1 # -- coding: utf-8 --
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	2 # :-
323 48430941c18c Adopt copyright and license wordings from https://reuse.software/faq/. Franz Glasner <fzglas.hg@dom66.de> parents: 322 diff changeset	3 # SPDX-FileCopyrightText: © 2025 Franz Glasner
48430941c18c Adopt copyright and license wordings from https://reuse.software/faq/. Franz Glasner <fzglas.hg@dom66.de> parents: 322 diff changeset	4 # SPDX-License-Identifier: BSD-3-Clause
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	5 # :-
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	6 r"""Generate and verify checksums for directory trees.
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	7
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	8 """
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	9
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	10 from __future__ import print_function, absolute_import
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	11
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	12
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	13 __all__ = []
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	14
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	15
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	16 import argparse
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	17 import base64
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	18 import binascii
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	19 import collections
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	20 import datetime
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	21 import errno
176 7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	22 import logging
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	23 import os
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	24 import re
157 27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	25 import stat
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	26 import sys
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	27 import time
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	28
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	29 from . import (__version__, __revision__)
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	30 from . import util
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	31 from .util import cm
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	32 from .util import digest
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	33 from .util import fnmatch
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	34 from .util import walk
260 07a0bc723139 treesum: Implement the CRC-32 using the new util.crc32 module Franz Glasner <fzglas.hg@dom66.de> parents: 223 diff changeset	35 from .util.crc32 import crc32
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	36
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	37
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	38 def main(argv=None):
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	39
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	40 def _populate_generate_arguments(gp):
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	41 """Use to populate command aliases.
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	42
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	43 This is because :class:`argparse.ArgumentParser` does not
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	44 support them for all supported Python versions.
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	45
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	46 """
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	47 gp.add_argument(
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	48 "--accept-treesum", "-A", action=PatternMatchAction,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	49 kind="accept-treesum",
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	50 dest="fnmatch_filters", metavar="PATTERN",
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	51 help="""Accept an existing treesum file PATTERN for a directory
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	52 checksum.
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	53 Implicitly this also acts as `--exclude' option.
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	54 Can be given more than once.""")
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	55 gp.add_argument(
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	56 "--algorithm", "-a", action="store", type=util.argv2algo,
172 804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	57 help="1 (aka sha1), 224, 256 (aka sha256), 384, 512 (aka sha512), "
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	58 "3 (alias for sha3-512), 3-224, 3-256, 3-384, 3-512, "
172 804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	59 "blake2b, blake2b-256, blake2s, "
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	60 "blake2 (alias for blake2b), "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	61 "blake2-256 (alias for blake2b-256), "
172 804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	62 "md5. "
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	63 "The default depends on availability in hashlib: "
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	64 "blake2b-256, sha256 or sha1.")
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	65 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	66 "--append-output", action="store_true", dest="append_output",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	67 help="Append to the output file instead of overwriting it.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	68 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	69 "--base64", action="store_true",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	70 help="Output checksums in base64 notation, not hexadecimal "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	71 "(OpenBSD).")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	72 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	73 "--comment", action="append", default=[],
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	74 help="Put given comment COMMENT into the output as \"COMMENT\". "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	75 "Can be given more than once.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	76 gp.add_argument(
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	77 "--exclude", "-X", action=PatternMatchAction, kind="exclude",
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	78 dest="fnmatch_filters", metavar="PATTERN",
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	79 help="""Exclude names matching the given PATTERN.
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	80 For help on PATTERN use \"help patterns\".
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	81 Can be given more than once.""")
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	82 gp.add_argument(
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	83 "--follow-directory-symlinks", "-l", action=SymlinkAction,
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	84 const="follow-directory-symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	85 default=FollowSymlinkConfig(False, False, True),
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	86 dest="follow_symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	87 help="""Follow symbolic links to directories when walking a
271 6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	88 directory tree. Augments --physical, --half and -p.""")
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	89 gp.add_argument(
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	90 "--follow-file-symlinks", action=SymlinkAction,
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	91 const="follow-file-symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	92 dest="follow_symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	93 help="""Follow symbolic links to files when walking a
271 6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	94 directory tree. Augments --physical and --half.""")
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	95 gp.add_argument(
158 d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	96 "--full-mode", action="store_true", dest="metadata_full_mode",
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	97 help="Consider all mode bits as returned from stat(2) when "
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	98 "computing directory digests. "
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	99 "Note that mode bits on symbolic links itself are not "
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	100 "considered.")
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	101 gp.add_argument(
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	102 "--generator", choices=("normal", "full", "none"),
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	103 default="normal",
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	104 help="""Put a `GENERATOR' line into the output.
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	105 `full' prints full Python and OS/platform version information,
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	106 `normal' prints just whether Python 2 or Python 3 is used, and `none'
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	107 suppresses the output completely. The default is `normal'.""")
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	108 gp.add_argument(
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	109 "--grouping-separator", action="store", dest="grouping_separator",
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	110 metavar="GROUPING-SEPARATOR",
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	111 help="""
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	112 Use the given GROUPING-SEPARATOR as thousands separator.
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	113 Use an empty GROUPING-SEPARATOR to disable grouping.
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	114 The effective default depends on the --output-style: for tagged output
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	115 it is the underscore `_', for tabbed output it is the dot `.'.
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	116
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	117 """)
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	118 gp.add_argument(
271 6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	119 "--half", "-H", action=SymlinkAction, dest="follow_symlinks",
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	120 const=FollowSymlinkConfig(True, False, False),
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	121 help="""Follow symbolic links given on the command line but do
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	122 not follow symlinks while traversing the directory tree.
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	123 Overwrites any other symlink related options
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	124 (--physical, --logical, -p, --no-follow-directory-symlinks,
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	125 --no-follow-file-symlinks, et al.).""")
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	126 gp.add_argument(
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	127 "--include", "-I", action=PatternMatchAction, kind="include",
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	128 dest="fnmatch_filters", metavar="PATTERN",
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	129 help="""Include names matching the given PATTERN.
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	130 For help on PATTERN use \"help patterns\".
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	131 Can be given more than once.""")
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	132 gp.add_argument(
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	133 "--logical", "-L", action=SymlinkAction, dest="follow_symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	134 const=FollowSymlinkConfig(True, True, True),
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	135 help="""Follow symbolic links everywhere: on command line
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	136 arguments and -- while walking -- directory and file symbolic links.
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	137 Overwrites any other symlink related options
271 6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	138 (--physical, --half, -p, --no-follow-directory-symlinks,
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	139 --no-follow-file-symlinks, et al.).""")
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	140 gp.add_argument(
150 f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	141 "--minimal", nargs="?", const="", default=None, metavar="TAG",
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	142 help="Produce minimal output only. If a TAG is given and not "
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	143 "empty use it as the leading \"ROOT (<TAG>)\" output.")
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	144 gp.add_argument(
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	145 "--mmap", action="store_true", dest="mmap", default=None,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	146 help="Use mmap if available. Default is to determine "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	147 "automatically from the filesize.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	148 gp.add_argument(
157 27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	149 "--mode", action="store_true", dest="metadata_mode",
158 d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	150 help="Consider the permission bits of stat(2) using S_IMODE (i.e. "
157 27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	151 "all bits without the filetype bits) when "
158 d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	152 "computing directory digests. Note that mode bits on "
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	153 "symbolic links itself are not considered.")
157 27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	154 gp.add_argument(
151 b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	155 "--mtime", action="store_true", dest="metadata_mtime",
b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	156 help="Consider the mtime of files (non-directories) when "
b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	157 "generating digests for directories. Digests for files are "
b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	158 "not affected.")
b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	159 gp.add_argument(
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	160 "--no-follow-directory-symlinks", action=SymlinkAction,
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	161 const="no-follow-directory-symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	162 dest="follow_symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	163 help="""Do not follow symbolic links to directories when walking a
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	164 directory tree. Augments --logical.""")
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	165 gp.add_argument(
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	166 "--no-follow-file-symlinks", action=SymlinkAction,
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	167 const="no-follow-file-symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	168 dest="follow_symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	169 help="""Dont follow symbolic links to files when walking a
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	170 directory tree. Augments --logical and -p.""")
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	171 gp.add_argument(
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	172 "--no-mmap", action="store_false", dest="mmap", default=None,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	173 help="Dont use mmap.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	174 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	175 "--output", "-o", action="store", metavar="OUTPUT",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	176 help="Put the checksum into given file. "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	177 "If not given or if it is given as `-' then stdout is used.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	178 gp.add_argument(
327 8e62738e7469 treesum: implement option "--output-style". Franz Glasner <fzglas.hg@dom66.de> parents: 326 diff changeset	179 "--output-style", dest="output_style", default="tagged",
330 61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	180 choices=("tagged", "tag", "tabular", "tab"),
327 8e62738e7469 treesum: implement option "--output-style". Franz Glasner <fzglas.hg@dom66.de> parents: 326 diff changeset	181 help="""
330 61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	182 Select the output style: "tagged" or "tag" selects a more BSD style tagged
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	183 format. "tabular" or "tab" select a more GNU style tabular format.
327 8e62738e7469 treesum: implement option "--output-style". Franz Glasner <fzglas.hg@dom66.de> parents: 326 diff changeset	184 Default is "tagged".
8e62738e7469 treesum: implement option "--output-style". Franz Glasner <fzglas.hg@dom66.de> parents: 326 diff changeset	185 """)
8e62738e7469 treesum: implement option "--output-style". Franz Glasner <fzglas.hg@dom66.de> parents: 326 diff changeset	186 gp.add_argument(
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	187 "--physical", "-P", action=SymlinkAction, dest="follow_symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	188 const=FollowSymlinkConfig(False, False, False),
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	189 help="""Do not follow any symbolic links whether they are given
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	190 on the command line or when walking the directory tree.
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	191 Overwrites any other symlink related options
271 6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	192 (--logical, --half, -p, --follow-directory-symlinks, --follow-file-symlinks,
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	193 et al.).""")
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	194 gp.add_argument(
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	195 "-p", action=SymlinkAction, dest="follow_symlinks",
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	196 const=FollowSymlinkConfig(False, False, True),
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	197 help="""Do not follow any symbolic links to directories,
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	198 whether they are given on the command line or when walking the directory tree,
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	199 but follow symbolic links to files.
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	200 Overwrites any other symlink related options
271 6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	201 (--logical, --half, --physical, --follow-directory-symlinks,
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	202 --no-follow-file-symlinks, et al.).
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	203 This is the default.""")
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	204 gp.add_argument(
168 bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	205 "--print-size", action="store_true",
bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	206 help="""Print the size of a file or the accumulated sizes of
bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	207 directory content into the output also.
bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	208 The size is not considered when computing digests. For symbolic links
bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	209 the size is not printed also.""")
bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	210 gp.add_argument(
170 8945be6b404e Mode for treesum.py to print only the size of files and the accumulated size of a directory: --size-only. Franz Glasner <fzglas.hg@dom66.de> parents: 169 diff changeset	211 "--size-only", action="store_true",
8945be6b404e Mode for treesum.py to print only the size of files and the accumulated size of a directory: --size-only. Franz Glasner <fzglas.hg@dom66.de> parents: 169 diff changeset	212 help="""Print only the size of files and for each directory its
8945be6b404e Mode for treesum.py to print only the size of files and the accumulated size of a directory: --size-only. Franz Glasner <fzglas.hg@dom66.de> parents: 169 diff changeset	213 accumulated directory size. Digests are not computed.""")
8945be6b404e Mode for treesum.py to print only the size of files and the accumulated size of a directory: --size-only. Franz Glasner <fzglas.hg@dom66.de> parents: 169 diff changeset	214 gp.add_argument(
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	215 "--size-width", action="store", type=int, metavar="SIZE-WIDTH",
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	216 dest="size_column_width", default=15,
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	217 help="""Some output styles print the a filesize right-aligned
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	218 in a column. SIZE-WIDTH is the (minimum) width to be used. The width includes
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	219 grouping separators. Use 0 if no alignment should be done. Default is 15.""")
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	220 gp.add_argument(
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	221 "--utf8", "--utf-8", action="store_true",
b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	222 help="""Encode all file paths using UTF-8 instead of
b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	223 the filesystem encoding. Add some error tag into the path if it cannot
b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	224 representated in Unicode cleanly.""")
b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	225 gp.add_argument(
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	226 "directories", nargs="*", metavar="DIRECTORY")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	227
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	228 def _populate_info_arguments(ip):
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	229 ip.add_argument(
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	230 "--last", action="store_true", dest="print_only_last_block",
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	231 help="Print only the last block of every given input file")
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	232 ip.add_argument(
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	233 "digest_files", nargs="+", metavar="TREESUM-DIGEST-FILE")
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	234
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	235 parser = argparse.ArgumentParser(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	236 description="Generate and verify checksums for directory trees.",
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	237 fromfile_prefix_chars='@',
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	238 add_help=False)
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	239
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	240 #
153 3505406ef9f3 Comment Franz Glasner <fzglas.hg@dom66.de> parents: 152 diff changeset	241 # Global options for all sub-commands.
3505406ef9f3 Comment Franz Glasner <fzglas.hg@dom66.de> parents: 152 diff changeset	242 # In a group because this allows a customized title.
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	243 #
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	244 gparser = parser.add_argument_group(title="Global Options")
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	245 gparser.add_argument(
191 1b8bc876146a Make "--debug" a global argument in treeview Franz Glasner <fzglas.hg@dom66.de> parents: 190 diff changeset	246 "--debug", action="store_true",
1b8bc876146a Make "--debug" a global argument in treeview Franz Glasner <fzglas.hg@dom66.de> parents: 190 diff changeset	247 help="Activate debug logging to stderr")
1b8bc876146a Make "--debug" a global argument in treeview Franz Glasner <fzglas.hg@dom66.de> parents: 190 diff changeset	248 gparser.add_argument(
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	249 "-v", "--version", action="version",
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	250 version="%s (rv:%s)" % (__version__, __revision__),
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	251 help="Show program's version number and exit")
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	252 gparser.add_argument(
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	253 "-h", "--help", action="help",
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	254 help="Show this help message and exit")
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	255
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	256 #
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	257 # Subcommands
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	258 #
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	259 subparsers = parser.add_subparsers(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	260 dest="subcommand",
145 073e0faea599 Optimize help output for subcommands Franz Glasner <fzglas.hg@dom66.de> parents: 144 diff changeset	261 title="Commands",
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	262 description="This tool uses subcommands. "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	263 "To see detailed help for a specific subcommand use "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	264 "the -h/--help option after the subcommand name. "
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	265 "A list of valid commands and their short descriptions "
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	266 "is listed below:",
145 073e0faea599 Optimize help output for subcommands Franz Glasner <fzglas.hg@dom66.de> parents: 144 diff changeset	267 metavar="COMMAND")
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	268
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	269 genparser = subparsers.add_parser(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	270 "generate",
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	271 help="Generate checksums for directory trees",
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	272 description="Generate checksums for directory trees.")
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	273 _populate_generate_arguments(genparser)
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	274 # And an alias for "generate"
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	275 genparser2 = subparsers.add_parser(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	276 "gen",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	277 help="Alias for \"generate\"",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	278 description="Generate checksums for directory trees. "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	279 "This is an alias to \"generate\".")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	280 _populate_generate_arguments(genparser2)
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	281
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	282 infoparser = subparsers.add_parser(
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	283 "info",
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	284 help="Print some information from given treesum digest file",
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	285 description="""Print some informations from given treesum digest files
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	286 to stdout."""
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	287 )
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	288 _populate_info_arguments(infoparser)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	289
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	290 hparser = subparsers.add_parser(
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	291 "help",
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	292 help="Show this help message or a subcommand's help and exit",
347 3f4840eeb0f4 treesum: optimize help output: do not print "-h/--help" for "patterns" and "help" Franz Glasner <fzglas.hg@dom66.de> parents: 346 diff changeset	293 description="Show this help message or a subcommand's help and exit.",
3f4840eeb0f4 treesum: optimize help output: do not print "-h/--help" for "patterns" and "help" Franz Glasner <fzglas.hg@dom66.de> parents: 346 diff changeset	294 add_help=False)
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	295 hparser.add_argument("help_command", nargs='?', metavar="COMMAND")
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	296
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	297 patparser = subparsers.add_parser(
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	298 "patterns",
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	299 help="Show the help for PATTERNs and exit",
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	300 description=fnmatch.HELP_DESCRIPTION,
347 3f4840eeb0f4 treesum: optimize help output: do not print "-h/--help" for "patterns" and "help" Franz Glasner <fzglas.hg@dom66.de> parents: 346 diff changeset	301 formatter_class=argparse.RawDescriptionHelpFormatter,
3f4840eeb0f4 treesum: optimize help output: do not print "-h/--help" for "patterns" and "help" Franz Glasner <fzglas.hg@dom66.de> parents: 346 diff changeset	302 add_help=False)
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	303
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	304 vparser = subparsers.add_parser(
144 b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	305 "version",
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	306 help="Show the program's version number and exit",
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	307 description="Show the program's version number and exit.")
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	308
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	309 # Parse leniently to just check for "version" and/or help
146 7d8df8311e3b Optimize argument parsing for the "version" command Franz Glasner <fzglas.hg@dom66.de> parents: 145 diff changeset	310 opts, _dummy = parser.parse_known_args(args=argv)
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	311
144 b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	312 if opts.subcommand == "version":
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	313 print("%s (rv:%s)" % (__version__, __revision__),
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	314 file=sys.stdout)
174 fc1055878775 Use "return 0" instead of "sys.exit(0)" when printing version and help# Franz Glasner <fzglas.hg@dom66.de> parents: 173 diff changeset	315 return 0
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	316 if opts.subcommand == "help":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	317 if not opts.help_command:
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	318 parser.print_help()
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	319 else:
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	320 if opts.help_command == "generate":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	321 genparser.print_help()
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	322 elif opts.help_command == "gen":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	323 genparser2.print_help()
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	324 elif opts.help_command == "info":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	325 infoparser.print_help()
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	326 elif opts.help_command == "version":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	327 vparser.print_help()
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	328 elif opts.help_command == "help":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	329 hparser.print_help()
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	330 elif opts.help_command == "patterns":
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	331 patparser.print_help()
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	332 else:
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	333 parser.print_help()
174 fc1055878775 Use "return 0" instead of "sys.exit(0)" when printing version and help# Franz Glasner <fzglas.hg@dom66.de> parents: 173 diff changeset	334 return 0
346 d47965f97abb treesum: Now handle all help related stuff in main() Franz Glasner <fzglas.hg@dom66.de> parents: 344 diff changeset	335 elif opts.subcommand == "patterns":
d47965f97abb treesum: Now handle all help related stuff in main() Franz Glasner <fzglas.hg@dom66.de> parents: 344 diff changeset	336 patparser.print_help()
d47965f97abb treesum: Now handle all help related stuff in main() Franz Glasner <fzglas.hg@dom66.de> parents: 344 diff changeset	337 return 0
144 b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	338
146 7d8df8311e3b Optimize argument parsing for the "version" command Franz Glasner <fzglas.hg@dom66.de> parents: 145 diff changeset	339 # Reparse strictly
7d8df8311e3b Optimize argument parsing for the "version" command Franz Glasner <fzglas.hg@dom66.de> parents: 145 diff changeset	340 opts = parser.parse_args(args=argv)
7d8df8311e3b Optimize argument parsing for the "version" command Franz Glasner <fzglas.hg@dom66.de> parents: 145 diff changeset	341
176 7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	342 # Minimal logging -- just for debugging - not for more "normal" use
7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	343 logging.basicConfig(
7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	344 level=logging.DEBUG if opts.debug else logging.WARNING,
7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	345 stream=sys.stderr,
7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	346 format="[%(asctime)s][%(levelname)s][%(process)d:%(name)s] %(message)s"
7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	347 )
7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	348 logging.captureWarnings(True)
7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	349
346 d47965f97abb treesum: Now handle all help related stuff in main() Franz Glasner <fzglas.hg@dom66.de> parents: 344 diff changeset	350 return treesum(opts)
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	351
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	352
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	353 FollowSymlinkConfig = collections.namedtuple("FollowSymlinkConfig",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	354 ["command_line",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	355 "directory",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	356 "file"])
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	357
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	358
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	359 class SymlinkAction(argparse.Action):
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	360
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	361 """`type' is fixed here.
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	362 `dest' is a tuple with three items:
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	363
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	364 1. follow symlinks on the command line
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	365 2. follow directory symlinks while walking
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	366 3. follow file symlinks while walking (not yet implemented)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	367
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	368 """
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	369
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	370 def __init__(self, args, *kwargs):
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	371 if "nargs" in kwargs:
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	372 raise ValueError("`nargs' not allowed")
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	373 if "type" in kwargs:
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	374 raise ValueError("`type' not allowed")
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	375 c = kwargs.get("const", None)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	376 if c is None:
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	377 raise ValueError("a const value is needed")
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	378 if (not isinstance(c, FollowSymlinkConfig)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	379 and c not in ("follow-directory-symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	380 "no-follow-directory-symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	381 "follow-file-symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	382 "no-follow-file-symlinks")):
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	383 raise ValueError(
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	384 "invalid value for the `const' configuration value")
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	385 default = kwargs.get("default", None)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	386 if (default is not None
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	387 and not isinstance(default, FollowSymlinkConfig)):
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	388 raise TypeError("invalid type for `default'")
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	389 kwargs["nargs"] = 0
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	390 super(SymlinkAction, self).__init__(args, *kwargs)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	391
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	392 def __call__(self, parser, namespace, values, option_string=None):
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	393 curval = getattr(namespace, self.dest, None)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	394 if curval is None:
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	395 curval = FollowSymlinkConfig(False, False, True)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	396 if isinstance(self.const, FollowSymlinkConfig):
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	397 curval = self.const
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	398 else:
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	399 if self.const == "follow-directory-symlinks":
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	400 curval = FollowSymlinkConfig(
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	401 curval.command_line, True, curval.file)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	402 elif self.const == "no-follow-directory-symlinks":
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	403 curval = FollowSymlinkConfig(
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	404 curval.command_line, False, curval.file)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	405 elif self.const == "follow-file-symlinks":
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	406 curval = FollowSymlinkConfig(
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	407 curval.command_line, curval.directory, True)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	408 elif self.const == "no-follow-file-symlinks":
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	409 curval = FollowSymlinkConfig(
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	410 curval.command_line, curval.directory, False)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	411 else:
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	412 assert False, "Implementation error: not yet implemented"
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	413
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	414 setattr(namespace, self.dest, curval)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	415
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	416
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	417 class PatternMatchAction(argparse.Action):
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	418
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	419 def __init__(self, args, *kwargs):
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	420 if "nargs" in kwargs:
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	421 raise argparse.ArgumentError(None, "`nargs' not allowed")
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	422 if "type" in kwargs:
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	423 raise argparse.ArgumentError(None, "`type' not allowed")
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	424 kwargs["nargs"] = 1
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	425
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	426 self.__kind = kwargs.pop("kind", None)
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	427 if self.__kind is None:
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	428 raise argparse.ArgumentError(None, "`kind' is required")
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	429 if self.__kind not in ("exclude", "include", "accept-treesum"):
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	430 raise argparse.ArgumentError(
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	431 None,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	432 "`kind' must be one of `include', `exclude' or"
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	433 " `accept-treesum'")
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	434
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	435 super(PatternMatchAction, self).__init__(args, *kwargs)
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	436
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	437 def __call__(self, parser, namespace, values, option_string=None):
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	438 items = getattr(namespace, self.dest, None)
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	439 if items is None:
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	440 items = []
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	441 setattr(namespace, self.dest, items)
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	442 for v in values:
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	443 items.append((self.__kind, v))
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	444
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	445
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	446 def gen_generate_opts(directories=[],
172 804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	447 algorithm=util.default_algotag(),
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	448 append_output=False,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	449 base64=False,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	450 comment=[],
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	451 fnmatch_filters=[],
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	452 follow_symlinks=FollowSymlinkConfig(False, False, False),
158 d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	453 full_mode=False,
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	454 generator="normal",
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	455 grouping_separator=None, # the output writer selects
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	456 logical=None,
150 f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	457 minimal=None,
157 27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	458 mode=False,
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	459 mmap=None,
151 b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	460 mtime=False,
168 bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	461 output=None,
327 8e62738e7469 treesum: implement option "--output-style". Franz Glasner <fzglas.hg@dom66.de> parents: 326 diff changeset	462 output_style="tagged",
170 8945be6b404e Mode for treesum.py to print only the size of files and the accumulated size of a directory: --size-only. Franz Glasner <fzglas.hg@dom66.de> parents: 169 diff changeset	463 print_size=False,
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	464 size_only=False,
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	465 size_column_width=15,
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	466 utf8=False):
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	467 if not isinstance(follow_symlinks, FollowSymlinkConfig):
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	468 raise TypeError("`follow_symlinks' must be a FollowSymlinkConfig")
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	469 if not isinstance(fnmatch_filters, (list, tuple, type(None))):
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	470 raise TypeError("`fnmatch_filters' must be a sequence (list, tuple)")
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	471 if fnmatch_filters:
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	472 for f in fnmatch_filters:
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	473 if not isinstance(f, (tuple, list)):
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	474 raise TypeError(
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	475 "items in `fnmatch_filters' must be tuples or lists")
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	476 if f[0] not in ("exclude", "include"):
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	477 raise ValueError(
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	478 "every kind of every item in `fnmatch_filters' must be"
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	479 " \"include\", \"exclude\" or \"accept-treesum\""
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	480 )
328 97bba2295eb3 treesum: parameter validation for "generator" Franz Glasner <fzglas.hg@dom66.de> parents: 327 diff changeset	481 if generator not in ("normal", "full", "none"):
97bba2295eb3 treesum: parameter validation for "generator" Franz Glasner <fzglas.hg@dom66.de> parents: 327 diff changeset	482 raise ValueError("given generator `%s' not allowed" % (generator, ))
330 61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	483 if output_style not in ("tagged", "tag", "tabular", "tab"):
327 8e62738e7469 treesum: implement option "--output-style". Franz Glasner <fzglas.hg@dom66.de> parents: 326 diff changeset	484 raise ValueError(
8e62738e7469 treesum: implement option "--output-style". Franz Glasner <fzglas.hg@dom66.de> parents: 326 diff changeset	485 "given output_style `%s' not allowed" % (output_style,))
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	486
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	487 # Not following symlinks to files is not yet supported: reset to True
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	488 # if not follow_symlinks.file:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	489 # follow_symlinks = follow_symlinks._make([follow_symlinks.command_line,
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	490 # follow_symlinks.directory,
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	491 # True])
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	492 # logging.warning("Coercing to follow-symlinks-file")
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	493 opts = argparse.Namespace(
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	494 directories=directories,
172 804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	495 algorithm=util.argv2algo(algorithm),
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	496 append_output=append_output,
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	497 base64=base64,
135 dbf27681a1f6 Allow to put comments into the output with "--comment" Franz Glasner <fzglas.hg@dom66.de> parents: 134 diff changeset	498 comment=comment,
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	499 fnmatch_filters=fnmatch_filters,
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	500 follow_symlinks=follow_symlinks,
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	501 generator=generator,
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	502 grouping_separator=grouping_separator,
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	503 logical=logical,
150 f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	504 minimal=minimal,
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	505 mmap=mmap,
158 d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	506 metadata_full_mode=full_mode,
157 27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	507 metadata_mode=mode,
151 b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	508 metadata_mtime=mtime,
168 bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	509 output=output,
327 8e62738e7469 treesum: implement option "--output-style". Franz Glasner <fzglas.hg@dom66.de> parents: 326 diff changeset	510 output_style=output_style,
170 8945be6b404e Mode for treesum.py to print only the size of files and the accumulated size of a directory: --size-only. Franz Glasner <fzglas.hg@dom66.de> parents: 169 diff changeset	511 print_size=print_size,
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	512 size_only=size_only,
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	513 size_column_width=size_column_width,
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	514 utf8=utf8)
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	515 return opts
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	516
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	517
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	518 def gen_info_opts(digest_files=[], last=False):
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	519 opts = argparse.Namespace(
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	520 digest_files=digest_files,
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	521 print_only_last_block=last)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	522 return opts
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	523
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	524
346 d47965f97abb treesum: Now handle all help related stuff in main() Franz Glasner <fzglas.hg@dom66.de> parents: 344 diff changeset	525 def treesum(opts):
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	526 # XXX TBD: opts.check and opts.checklist (as in shasum.py)
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	527 if opts.subcommand in ("generate", "gen"):
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	528 return generate_treesum(opts)
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	529 elif opts.subcommand == "info":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	530 return print_treesum_digestfile_infos(opts)
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	531 else:
346 d47965f97abb treesum: Now handle all help related stuff in main() Franz Glasner <fzglas.hg@dom66.de> parents: 344 diff changeset	532 #
d47965f97abb treesum: Now handle all help related stuff in main() Franz Glasner <fzglas.hg@dom66.de> parents: 344 diff changeset	533 # NOTE: Subcommands for printing help (e.g. "patterns") should
d47965f97abb treesum: Now handle all help related stuff in main() Franz Glasner <fzglas.hg@dom66.de> parents: 344 diff changeset	534 # be handled in the caller.
d47965f97abb treesum: Now handle all help related stuff in main() Franz Glasner <fzglas.hg@dom66.de> parents: 344 diff changeset	535 #
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	536 raise RuntimeError(
346 d47965f97abb treesum: Now handle all help related stuff in main() Franz Glasner <fzglas.hg@dom66.de> parents: 344 diff changeset	537 "command `{}' not handled".format(opts.subcommand))
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	538
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	539
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	540 def generate_treesum(opts):
149 f717854be1de Put the defaults generation when generating directory digests into "generate_treesum()" instead of "main()" Franz Glasner <fzglas.hg@dom66.de> parents: 148 diff changeset	541 # Provide defaults
f717854be1de Put the defaults generation when generating directory digests into "generate_treesum()" instead of "main()" Franz Glasner <fzglas.hg@dom66.de> parents: 148 diff changeset	542 if not opts.algorithm:
172 804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	543 opts.algorithm = util.argv2algo(util.default_algotag())
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	544 if not opts.directories:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	545 opts.directories.append(".")
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	546
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	547 if opts.output is None or opts.output == "-":
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	548 if hasattr(sys.stdout, "buffer"):
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	549 out_cm = cm.nullcontext(sys.stdout.buffer)
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	550 else:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	551 out_cm = cm.nullcontext(sys.stdout)
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	552 else:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	553 if opts.append_output:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	554 out_cm = open(opts.output, "ab")
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	555 else:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	556 out_cm = open(opts.output, "wb")
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	557
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	558 fnmatcher = fnmatch.FnMatcher.build_from_commandline_patterns(
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	559 opts.fnmatch_filters)
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	560
330 61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	561 if opts.output_style in ("tagged", "tag"):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	562 writerstyle = TaggedTreesumWriter
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	563 elif opts.output_style in ("tabular", "tab"):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	564 writerstyle = TabularTreesumWriter
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	565 else:
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	566 raise NotImplementedError("`output_style'")
327 8e62738e7469 treesum: implement option "--output-style". Franz Glasner <fzglas.hg@dom66.de> parents: 326 diff changeset	567
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	568 with out_cm as outfp:
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	569 writer = writerstyle(outfp,
336 0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	570 size_only=opts.size_only,
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	571 print_size=opts.print_size,
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	572 use_base64=opts.base64,
341 728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	573 grouping_separator=opts.grouping_separator,
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	574 size_column_width=opts.size_column_width,
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	575 )
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	576 for d in opts.directories:
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	577 V1DirectoryTreesumGenerator(
331 9ee84624587f treesum: move the handling of base64 digest output into the writers completely Franz Glasner <fzglas.hg@dom66.de> parents: 330 diff changeset	578 opts.algorithm, opts.mmap,
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	579 opts.follow_symlinks,
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	580 opts.generator,
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	581 opts.metadata_mode,
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	582 opts.metadata_full_mode,
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	583 opts.metadata_mtime,
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	584 opts.size_only,
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	585 opts.print_size,
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	586 opts.utf8,
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	587 minimal=opts.minimal,
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	588 fnmatcher=fnmatcher).generate(
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	589 writer, d, comment=opts.comment)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	590
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	591
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	592 class V1DirectoryTreesumGenerator(object):
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	593
331 9ee84624587f treesum: move the handling of base64 digest output into the writers completely Franz Glasner <fzglas.hg@dom66.de> parents: 330 diff changeset	594 def __init__(self, algorithm, use_mmap,
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	595 follow_symlinks,
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	596 with_generator,
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	597 with_metadata_mode, with_metadata_full_mode,
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	598 with_metadata_mtime, size_only, print_size, utf8_mode,
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	599 minimal=None,
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	600 fnmatcher=None):
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	601 super(V1DirectoryTreesumGenerator, self).__init__()
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	602 self._algorithm = algorithm
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	603 self._use_mmap = use_mmap
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	604 self._follow_symlinks = follow_symlinks
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	605 self._with_generator = with_generator
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	606 self._with_metadata_mode = with_metadata_mode
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	607 self._with_metadata_full_mode = with_metadata_full_mode
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	608 self._with_metadata_mtime = with_metadata_mtime
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	609 self._size_only = size_only
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	610 self._print_size = print_size
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	611 self._utf8_mode = utf8_mode
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	612 self._minimal = minimal
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	613 self._fnmatcher = fnmatcher
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	614
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	615 def generate(self, writer, root, comment=None):
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	616 """
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	617
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	618 :param outfp: a binary file with a "write()" and a "flush()" method
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	619
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	620 """
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	621 self._writer = writer
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	622 self._writer.start("1")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	623 self._writer.write_fsencoding(util.n(walk.getfsencoding().upper()))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	624 self._writer.flush()
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	625
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	626 if self._with_generator == "none":
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	627 pass # do nothing
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	628 elif self._with_generator == "normal":
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	629 self._writer.write_generator("PY2" if util.PY2 else "PY3")
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	630 elif self._with_generator == "full":
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	631 import platform
295 4a259fb9968e treesum: Add the treesum version and revision to the full generator output Franz Glasner <fzglas.hg@dom66.de> parents: 291 diff changeset	632 info = ("treesum %s (rv:%s), %s %s, %s"
4a259fb9968e treesum: Add the treesum version and revision to the full generator output Franz Glasner <fzglas.hg@dom66.de> parents: 291 diff changeset	633 % (__version__,
4a259fb9968e treesum: Add the treesum version and revision to the full generator output Franz Glasner <fzglas.hg@dom66.de> parents: 291 diff changeset	634 __revision__,
4a259fb9968e treesum: Add the treesum version and revision to the full generator output Franz Glasner <fzglas.hg@dom66.de> parents: 291 diff changeset	635 platform.python_implementation(),
4a259fb9968e treesum: Add the treesum version and revision to the full generator output Franz Glasner <fzglas.hg@dom66.de> parents: 291 diff changeset	636 platform.python_version(),
4a259fb9968e treesum: Add the treesum version and revision to the full generator output Franz Glasner <fzglas.hg@dom66.de> parents: 291 diff changeset	637 platform.platform()))
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	638 self._writer.write_generator(info)
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	639 else:
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	640 raise NotImplementedError(
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	641 "not implemented: %s" % (self._with_generator,))
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	642
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	643 #
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	644 # Note: Given non-default flags that are relevant for
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	645 # directory traversal.
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	646 #
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	647 flags = []
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	648 if self._with_metadata_full_mode:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	649 flags.append("with-metadata-fullmode")
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	650 elif self._with_metadata_mode:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	651 flags.append("with-metadata-mode")
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	652 if self._with_metadata_mtime:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	653 flags.append("with-metadata-mtime")
219 19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	654 flags.append("follow-symlinks-commandline"
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	655 if self._follow_symlinks.command_line
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	656 else "no-follow-symlinks-commandline")
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	657 flags.append("follow-symlinks-directory"
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	658 if self._follow_symlinks.directory
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	659 else "no-follow-symlinks-directory")
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	660 flags.append("follow-symlinks-file"
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	661 if self._follow_symlinks.file
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	662 else "no-follow-symlinks-file")
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	663 if self._size_only:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	664 flags.append("size-only")
219 19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	665 flags.append("utf8-encoding" if self._utf8_mode else "fs-encoding")
211 5bb0b25f8e99 FIX: FLAGS output: wrong "else" for "utf-8-mode" and "print-size" Franz Glasner <fzglas.hg@dom66.de> parents: 210 diff changeset	666 if self._print_size:
5bb0b25f8e99 FIX: FLAGS output: wrong "else" for "utf-8-mode" and "print-size" Franz Glasner <fzglas.hg@dom66.de> parents: 210 diff changeset	667 flags.append("print-size")
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	668 self._writer.write_flags(flags)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	669
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	670 if self._minimal is None:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	671 # Write execution timestamps in POSIX epoch and ISO format
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	672 ts = int(time.time())
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	673 self._writer.write_timestamp(ts)
291 cc2deeb5f2e6 treesum: Indicate that the ISO timestamp is UTC Franz Glasner <fzglas.hg@dom66.de> parents: 290 diff changeset	674 ts = (datetime.datetime.utcfromtimestamp(ts)).isoformat("T") + "Z"
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	675 self._writer.write_isotimestamp(ts)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	676
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	677 if comment:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	678 for line in comment:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	679 self._writer.write_comment(line)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	680
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	681 for action, kind, pattern in self._fnmatcher.definitions():
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	682 self._writer.write_fnmatch_pattern(action, kind, pattern)
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	683
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	684 if self._minimal is not None:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	685 self._writer.write_root(
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	686 (walk.WalkDirEntry.alt_u8(self._minimal)
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	687 if self._minimal else b""))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	688 else:
278 822cf3a1da22 treesum: FIX: Removal of backslashes in output: do this only where really needed. Franz Glasner <fzglas.hg@dom66.de> parents: 277 diff changeset	689 self._writer.write_root(walk.WalkDirEntry.alt_u8(
822cf3a1da22 treesum: FIX: Removal of backslashes in output: do this only where really needed. Franz Glasner <fzglas.hg@dom66.de> parents: 277 diff changeset	690 util.normalize_filename(root, True)))
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	691
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	692 self._writer.flush()
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	693
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	694 if not self._follow_symlinks.command_line and os.path.islink(root):
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	695 linktgt = walk.WalkDirEntry.from_readlink(os.readlink(root))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	696 linkdgst = self._algorithm[0]()
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	697 linkdgst.update(linktgt.fspath)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	698 dir_dgst = self._algorithm[0]()
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	699 dir_dgst.update(b"2:L@,")
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	700 dir_dgst.update(
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	701 util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	702 b"%d:%s,%d:%s,",
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	703 len(self._algorithm[1]), util.b(self._algorithm[1]),
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	704 len(linkdgst.digest()), linkdgst.digest()))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	705 if self._size_only:
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	706 self._writer.write_size(b"./@/", -1)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	707 else:
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	708 sz = -1 if self._print_size else None
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	709 self._writer.write_file_digest(
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	710 self._algorithm[1],
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	711 b"./@/",
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	712 dir_dgst.digest(),
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	713 sz)
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	714 self._writer.flush()
268 8aadffaaad5f treesum: refactor: remove early return and use an "else" branch instead Franz Glasner <fzglas.hg@dom66.de> parents: 267 diff changeset	715 else:
8aadffaaad5f treesum: refactor: remove early return and use an "else" branch instead Franz Glasner <fzglas.hg@dom66.de> parents: 267 diff changeset	716 self._generate(os.path.normpath(root), tuple())
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	717 self._writer.finish()
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	718
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	719 def _generate(self, root, top):
301 d246b631b85a treesum: Change debug output: make output when entering a new root directory different from recursing into a sub-directory Franz Glasner <fzglas.hg@dom66.de> parents: 300 diff changeset	720 if top:
d246b631b85a treesum: Change debug output: make output when entering a new root directory different from recursing into a sub-directory Franz Glasner <fzglas.hg@dom66.de> parents: 300 diff changeset	721 logging.debug("Recursing into directory: %s/%r", root, top)
d246b631b85a treesum: Change debug output: make output when entering a new root directory different from recursing into a sub-directory Franz Glasner <fzglas.hg@dom66.de> parents: 300 diff changeset	722 else:
d246b631b85a treesum: Change debug output: make output when entering a new root directory different from recursing into a sub-directory Franz Glasner <fzglas.hg@dom66.de> parents: 300 diff changeset	723 logging.debug("Handling root directory: %s", root)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	724 path = os.path.join(root, *top) if top else root
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	725 # Determine also the path to be used for directory filtering
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	726 fpath = join_output_path(top, None) if top else ""
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	727 if self._fnmatcher:
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	728 if not self._fnmatcher.shall_visit(fpath):
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	729 logging.debug("Skipping directory: %s", fpath)
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	730 return (None, None, None, None)
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	731 try:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	732 with walk.ScanDir(path) as dirscan:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	733 fsobjects = list(dirscan)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	734 except OSError as e:
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	735 #
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	736 # NOTE: Sync the error handler code with this method's
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	737 # code below before returning!
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	738 #
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	739 if e.errno == errno.ENOTDIR:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	740 # object exists but is not a directory
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	741 errmsg = b"not a directory"
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	742 elif e.errno in (errno.EACCES, errno.EPERM,
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	743 getattr(errno, "ENOTCAPABLE", errno.EACCES)):
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	744 # no permissions
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	745 errmsg = (
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	746 b"access denied / no permissions / missing capabilities")
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	747 elif e.errno == errno.ENOENT:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	748 # given object does not exist
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	749 errmsg = b"no such file or directory"
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	750 else:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	751 raise
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	752 if self._utf8_mode:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	753 opath = walk.WalkDirEntry.alt_u8(path)
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	754 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	755 opath = walk.WalkDirEntry.alt_fs(path)
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	756 self._writer.write_error(util.interpolate_bytes(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	757 b"`%s': %s", opath, errmsg))
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	758 # Reuse from top
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	759 opath = join_output_path(top, None)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	760 if opath:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	761 if self._utf8_mode:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	762 opath = walk.WalkDirEntry.alt_u8(opath)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	763 else:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	764 opath = walk.WalkDirEntry.alt_fs(opath)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	765 if self._size_only:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	766 self._writer.write_size(opath, None)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	767 else:
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	768 self._writer.write_file_digest(
e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	769 self._algorithm[1], opath, None, None)
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	770 self._writer.flush()
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	771 return (e.errno, None, None, None)
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	772
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	773 # Check whether to accept existing treesum digest files
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	774 if self._fnmatcher:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	775 for fso in fsobjects:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	776 fpath = join_output_path(top, fso.name)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	777 if self._fnmatcher.shall_accept_treesum(fpath):
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	778 # Yes we have hit a treesum digest file
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	779 logging.debug("Accepting existing treesum from: %s", fpath)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	780 collector = DigestSizeCollector()
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	781 try:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	782 collector.collect_from_file(os.path.join(root, fpath))
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	783 except OSError as e:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	784 eno = e.errno
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	785 emsg = e.strerror
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	786 except Exception as e:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	787 # XXX FIXME: other EIO, EBADF, EFAULT
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	788 eno = errno.ESRCH
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	789 emsg = str(e)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	790 else:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	791 eno = 0
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	792 emsg = None
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	793 if self._utf8_mode:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	794 fpath = walk.WalkDirEntry.alt_u8(fpath)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	795 else:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	796 fpath = walk.WalkDirEntry.alt_fs(fpath)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	797 opath = join_output_path(top, None)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	798 if self._utf8_mode:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	799 opath = walk.WalkDirEntry.alt_u8(opath)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	800 else:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	801 opath = walk.WalkDirEntry.alt_fs(opath)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	802 if eno == 0:
313 dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	803 #
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	804 # treesum file could be read.
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	805 # Now check whether the infos we got from it are
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	806 # compatible with our current requirements
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	807 # (digest, size).
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	808 #
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	809 if self._size_only:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	810 if collector.size is None:
313 dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	811 #
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	812 # This is a severe error here: just the size
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	813 # is required, but we have not got one.
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	814 #
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	815 self._writer.write_error(util.b(
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	816 util.interpolate_bytes(
315 aaf0b3b53e48 treesum: FIX: format string for interpolate_bytes() must be of bytes type Franz Glasner <fzglas.hg@dom66.de> parents: 314 diff changeset	817 b"No size in treesum-file `%s' while"
aaf0b3b53e48 treesum: FIX: format string for interpolate_bytes() must be of bytes type Franz Glasner <fzglas.hg@dom66.de> parents: 314 diff changeset	818 b" requiring it",
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	819 fpath),
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	820 "utf-8"))
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	821 self._writer.write_size(opath, None)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	822 return (errno.ESRCH, None, None, None)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	823 else:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	824 if self._print_size:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	825 if collector.size is None:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	826 #
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	827 # XXX FIXME: Is this a severe error
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	828 # here? Currently: no
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	829 #
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	830 self._writer.write_error(util.b(
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	831 util.interpolate_bytes(
315 aaf0b3b53e48 treesum: FIX: format string for interpolate_bytes() must be of bytes type Franz Glasner <fzglas.hg@dom66.de> parents: 314 diff changeset	832 b"No size in treesum-file `%s'",
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	833 fpath),
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	834 "utf-8"))
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	835 sz = -1
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	836 else:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	837 sz = collector.size
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	838 else:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	839 sz = None
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	840 if collector.digest is None:
313 dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	841 #
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	842 # This is really a severe error. Most probably
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	843 # the treesum file was created with
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	844 # "--size-only" and contains no digest.
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	845 #
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	846 self._writer.write_error(util.b(
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	847 util.interpolate_bytes(
315 aaf0b3b53e48 treesum: FIX: format string for interpolate_bytes() must be of bytes type Franz Glasner <fzglas.hg@dom66.de> parents: 314 diff changeset	848 b"No digest in treesum-file `%s' while"
aaf0b3b53e48 treesum: FIX: format string for interpolate_bytes() must be of bytes type Franz Glasner <fzglas.hg@dom66.de> parents: 314 diff changeset	849 b" it is required",
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	850 fpath),
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	851 "utf-8"))
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	852 self._writer.write_file_digest(
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	853 collector.algorithm or "MD5",
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	854 opath,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	855 None,
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	856 sz)
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	857 return (errno.ESRCH, None, None, None)
313 dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	858 # We got all required infos without errors
317 fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	859 self._writer.write_accept_treesum_file(fpath)
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	860 if self._size_only:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	861 self._writer.write_size(opath, collector.size)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	862 else:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	863 self._writer.write_file_digest(
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	864 collector.algorithm, opath, collector.digest,
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	865 sz)
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	866 return (0,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	867 collector.algorithm,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	868 collector.digest,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	869 collector.size)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	870 else:
313 dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	871 #
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	872 # treesum file could not be read
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	873 #
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	874 self._writer.write_error(util.interpolate_bytes(
315 aaf0b3b53e48 treesum: FIX: format string for interpolate_bytes() must be of bytes type Franz Glasner <fzglas.hg@dom66.de> parents: 314 diff changeset	875 b"Cannot read treesum-file `%s' for directory"
aaf0b3b53e48 treesum: FIX: format string for interpolate_bytes() must be of bytes type Franz Glasner <fzglas.hg@dom66.de> parents: 314 diff changeset	876 b"`%s': %s",
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	877 fpath,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	878 opath,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	879 util.b(emsg, "utf-8")))
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	880 if self._size_only:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	881 self._writer.write_size(opath, None)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	882 else:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	883 self._writer.write_file_digest(
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	884 self._algorithm[1], opath, None, None)
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	885 return (eno, None, None, None)
313 dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	886 #
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	887 # No treesum file: just process normally with digesting
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	888 #
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	889 if self._utf8_mode:
221 ca9d5a0dc9bb Rename WalkDirEntry.sort_key and WalkDirEntry.alt_sort_key Franz Glasner <fzglas.hg@dom66.de> parents: 220 diff changeset	890 fsobjects.sort(key=walk.WalkDirEntry.sort_key_u8)
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	891 else:
221 ca9d5a0dc9bb Rename WalkDirEntry.sort_key and WalkDirEntry.alt_sort_key Franz Glasner <fzglas.hg@dom66.de> parents: 220 diff changeset	892 fsobjects.sort(key=walk.WalkDirEntry.sort_key_fs)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	893 dir_dgst = self._algorithm[0]()
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	894 dir_size = 0
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	895 dir_tainted = False
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	896 for fso in fsobjects:
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	897 # Determine the effective name to be used for digesting
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	898 if self._utf8_mode:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	899 if fso.u8name is None:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	900 dir_tainted = True
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	901 effective_fso_name = fso.alt_u8name
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	902 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	903 effective_fso_name = fso.u8name
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	904 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	905 if fso.fsname is None:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	906 dir_tainted = True
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	907 effective_fso_name = fso.alt_fsname
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	908 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	909 effective_fso_name = fso.fsname
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	910 # Determine the path (mostly its prefix) that is to be printed
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	911 opath = join_output_path(top, fso.name)
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	912 # Determine the path to be used for filename filtering
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	913 fpath = opath
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	914 if self._fnmatcher:
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	915 if not self._fnmatcher.shall_visit(fpath):
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	916 logging.debug("Skipping: %s", fpath)
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	917 continue
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	918 if self._utf8_mode:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	919 opath = walk.WalkDirEntry.alt_u8(opath)
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	920 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	921 opath = walk.WalkDirEntry.alt_fs(opath)
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	922 if fso.is_special:
344 0a58948df713 Move the computation of the special tag string marker for special files into a property Franz Glasner <fzglas.hg@dom66.de> parents: 341 diff changeset	923 special_tag = util.b(fso.special_tag)
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	924 assert fso.stat is not None # because .is_special is True
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	925 if fso.is_symlink and not self._follow_symlinks.file:
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	926 linktgt = walk.WalkDirEntry.from_readlink(
b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	927 os.readlink(fso.path))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	928 linkdgst = self._algorithm[0]()
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	929 if self._utf8_mode:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	930 if linktgt.u8path is None:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	931 dir_tainted = True
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	932 linkdgst.update(linktgt.alt_u8path)
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	933 else:
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	934 linkdgst.update(linktgt.u8path)
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	935 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	936 if linktgt.fspath is None:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	937 dir_tainted = True
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	938 linkdgst.update(linktgt.alt_fspath)
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	939 else:
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	940 linkdgst.update(linktgt.fspath)
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	941 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	942 b"2:@%s,%d:%s,",
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	943 special_tag,
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	944 len(effective_fso_name),
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	945 effective_fso_name))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	946 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	947 b"%d:%s,%d:%s,",
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	948 len(self._algorithm[1]), util.b(self._algorithm[1]),
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	949 len(linkdgst.digest()), linkdgst.digest()))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	950 #
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	951 # - no mtime and no mode for symlinks
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	952 # - also does not count for dir_size
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	953 #
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	954 if self._size_only:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	955 self._writer.write_size(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	956 util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	957 b"%s/./@%s", opath, special_tag),
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	958 -1)
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	959 else:
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	960 sz = -1 if self._print_size else None
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	961 self._writer.write_file_digest(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	962 self._algorithm[1],
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	963 util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	964 b"%s/./@%s", opath, special_tag),
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	965 linkdgst.digest(),
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	966 sz)
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	967 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	968 #
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	969 # Follow the symlink to special file and/or handle a
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	970 # special file
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	971 #
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	972 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	973 b"1:%s,%d:%s,",
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	974 special_tag,
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	975 len(effective_fso_name),
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	976 effective_fso_name))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	977 # no important size here but a mode
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	978 if self._with_metadata_mtime:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	979 mtime = datetime.datetime.utcfromtimestamp(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	980 int(fso.stat.st_mtime))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	981 mtime = util.b(mtime.isoformat("T") + "Z")
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	982 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	983 b"5:mtime,%d:%s,", len(mtime), mtime))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	984 if self._with_metadata_full_mode:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	985 modestr = util.b(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	986 normalized_mode_str(fso.stat.st_mode))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	987 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	988 b"8:fullmode,%d:%s,", len(modestr), modestr))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	989 elif self._with_metadata_mode:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	990 modestr = util.b(normalized_compatible_mode_str(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	991 fso.stat.st_mode))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	992 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	993 b"4:mode,%d:%s,", len(modestr), modestr))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	994 if self._size_only:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	995 self._writer.write_size(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	996 util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	997 b"%s/./%s", opath, special_tag),
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	998 -1)
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	999 else:
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1000 sz = -1 if self._print_size else None
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1001 self._writer.write_file_digest(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1002 self._algorithm[1],
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1003 util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1004 b"%s/./%s", opath, special_tag),
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1005 b"",
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	1006 sz)
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1007 elif fso.is_dir:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1008 assert fso.stat is not None # because .is_dir is True
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1009 if fso.is_symlink and not self._follow_symlinks.directory:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1010 linktgt = walk.WalkDirEntry.from_readlink(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1011 os.readlink(fso.path))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1012 linkdgst = self._algorithm[0]()
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1013 if self._utf8_mode:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1014 if linktgt.u8path is None:
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1015 dir_tainted = True
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1016 linkdgst.update(linktgt.alt_u8path)
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1017 else:
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1018 linkdgst.update(linktgt.u8path)
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1019 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1020 if linktgt.fspath is None:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1021 dir_tainted = True
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1022 linkdgst.update(linktgt.alt_fspath)
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1023 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1024 linkdgst.update(linktgt.fspath)
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1025 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1026 b"2:@/,%d:%s,",
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1027 len(effective_fso_name),
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1028 effective_fso_name))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1029 #
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1030 # - no mtime and no mode for symlinks
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1031 # - also does not count for dir_size
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1032 #
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1033 dir_dgst.update(util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1034 b"%d:%s,%d:%s,",
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1035 len(self._algorithm[1]), util.b(self._algorithm[1]),
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1036 len(linkdgst.digest()), linkdgst.digest()))
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1037 if self._size_only:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1038 self._writer.write_size(
218 dee891ed2307 FIX: Output of symlinks was not converted property but was written in bytes repr Franz Glasner <fzglas.hg@dom66.de> parents: 217 diff changeset	1039 util.interpolate_bytes(b"%s/./@/", opath),
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1040 -1)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1041 else:
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1042 sz = -1 if self._print_size else None
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1043 self._writer.write_file_digest(
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1044 self._algorithm[1],
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1045 util.interpolate_bytes(b"%s/./@/", opath),
334 5afece258bf2 treesum: FIX: add missing size param Franz Glasner <fzglas.hg@dom66.de> parents: 333 diff changeset	1046 linkdgst.digest(),
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	1047 sz)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1048 else:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1049 #
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1050 # Follow the symlink to dir or handle a "real" directory
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1051 #
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1052
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1053 # Get subdir data from recursing into it
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1054 sub_dir_errno, sub_dir_algo, sub_dir_dgst, sub_dir_size = \
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1055 self._generate(root, top + (fso.name, ))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1056
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1057 #
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1058 # Check first whether the directory was selected to be
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1059 # excluded
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1060 #
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1061 if sub_dir_errno is None:
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1062 # Yes -- skipped
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1063 continue
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1064 if sub_dir_errno == 0:
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1065 if sub_dir_size is None:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1066 if self._print_size or self._size_only:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1067 dir_tainted = True
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1068 else:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1069 dir_size += (sub_dir_size or 0)
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1070 else:
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1071 dir_tainted = True
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1072 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1073 b"1:/,%d:%s,",
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1074 len(effective_fso_name),
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1075 effective_fso_name))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1076 if self._with_metadata_full_mode:
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1077 modestr = util.b(normalized_mode_str(fso.stat.st_mode))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1078 dir_dgst.update(util.interpolate_bytes(
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1079 b"8:fullmode,%d:%s,", len(modestr), modestr))
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1080 elif self._with_metadata_mode:
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1081 modestr = util.b(normalized_compatible_mode_str(
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1082 fso.stat.st_mode))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1083 dir_dgst.update(util.interpolate_bytes(
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1084 b"4:mode,%d:%s,", len(modestr), modestr))
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1085 if sub_dir_errno == 0:
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1086 dir_dgst.update(util.interpolate_bytes(
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1087 b"%d:%s,%d:%s,",
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1088 len(sub_dir_algo), util.b(sub_dir_algo),
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1089 len(sub_dir_dgst), sub_dir_dgst))
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1090 else:
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1091 # NOTE: error message is already printed here
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1092 dir_dgst.update(util.interpolate_bytes(
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1093 b"5:errno,%d:%s",
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1094 len(str(sub_dir_errno)),
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1095 util.b(str(sub_dir_errno))))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1096 else:
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1097 if fso.is_symlink and not self._follow_symlinks.file:
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1098 #
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1099 # Symbolic link to some filesystem object which is not
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1100 # determined to be a link to a directory or some other
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1101 # special file (socket, FIFO, et al.).
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1102 #
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1103 linktgt = walk.WalkDirEntry.from_readlink(
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1104 os.readlink(fso.path))
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1105 linkdgst = self._algorithm[0]()
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1106 if self._utf8_mode:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1107 if linktgt.u8path is None:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1108 dir_tainted = True
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1109 linkdgst.update(linktgt.alt_u8path)
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1110 else:
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1111 linkdgst.update(linktgt.u8path)
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1112 else:
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1113 if linktgt.fspath is None:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1114 dir_tainted = True
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1115 linkdgst.update(linktgt.alt_fspath)
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1116 else:
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1117 linkdgst.update(linktgt.fspath)
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1118 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1119 b"1:@,%d:%s,",
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1120 len(effective_fso_name),
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1121 effective_fso_name))
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1122 #
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1123 # - no mtime and no mode for symlinks
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1124 # - also does not count for dir_size
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1125 #
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1126 dir_dgst.update(util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1127 b"%d:%s,%d:%s,",
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1128 len(self._algorithm[1]), util.b(self._algorithm[1]),
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1129 len(linkdgst.digest()), linkdgst.digest()))
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1130 if self._size_only:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1131 self._writer.write_size(
218 dee891ed2307 FIX: Output of symlinks was not converted property but was written in bytes repr Franz Glasner <fzglas.hg@dom66.de> parents: 217 diff changeset	1132 util.interpolate_bytes(b"%s/./@", opath),
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1133 -1)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1134 else:
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1135 sz = -1 if self._print_size else None
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1136 self._writer.write_file_digest(
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1137 self._algorithm[1],
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1138 util.interpolate_bytes(b"%s/./@", opath),
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1139 linkdgst.digest(),
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	1140 sz)
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1141 else:
223 61e5b1c2685c Commentx Franz Glasner <fzglas.hg@dom66.de> parents: 221 diff changeset	1142 #
61e5b1c2685c Commentx Franz Glasner <fzglas.hg@dom66.de> parents: 221 diff changeset	1143 # Follow the symlink to file or handle a "real" file
61e5b1c2685c Commentx Franz Glasner <fzglas.hg@dom66.de> parents: 221 diff changeset	1144 #
61e5b1c2685c Commentx Franz Glasner <fzglas.hg@dom66.de> parents: 221 diff changeset	1145
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1146 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1147 b"0:,%d:%s,",
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1148 len(effective_fso_name),
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1149 effective_fso_name))
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1150 if fso.stat is None:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1151 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1152 # Error: most likely a broken symlink here
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1153 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1154 dir_tainted = True
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1155 dir_dgst.update(util.interpolate_bytes(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1156 b"5:errno,%d:%s,",
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1157 len(str(fso.stat_errno)),
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1158 util.b(str(fso.stat_errno))))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1159 self._writer.write_error(util.interpolate_bytes(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1160 b"errno %d: %s",
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1161 fso.stat_errno,
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1162 util.b(fso.stat_errstr, "utf-8")))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1163 logging.error(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1164 "Directory entry has symlink problems: %r",
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1165 opath)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1166 if self._size_only:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1167 self._writer.write_size(opath, None)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1168 else:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1169 self._writer.write_file_digest(
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	1170 self._algorithm[1], opath, None, None)
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1171 else:
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1172 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1173 # Ok: File has normal stat info
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1174 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1175 # XXX FIXME: Handle special files (fifo, socket,
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1176 # block or char devices, ...).
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1177 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1178 dir_size += fso.stat.st_size
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1179 if self._with_metadata_mtime:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1180 mtime = datetime.datetime.utcfromtimestamp(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1181 int(fso.stat.st_mtime))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1182 mtime = util.b(mtime.isoformat("T") + "Z")
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1183 dir_dgst.update(util.interpolate_bytes(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1184 b"5:mtime,%d:%s,", len(mtime), mtime))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1185 if self._with_metadata_full_mode:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1186 modestr = util.b(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1187 normalized_mode_str(fso.stat.st_mode))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1188 dir_dgst.update(util.interpolate_bytes(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1189 b"8:fullmode,%d:%s,", len(modestr), modestr))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1190 elif self._with_metadata_mode:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1191 modestr = util.b(normalized_compatible_mode_str(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1192 fso.stat.st_mode))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1193 dir_dgst.update(util.interpolate_bytes(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1194 b"4:mode,%d:%s,", len(modestr), modestr))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1195 if self._size_only:
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1196 #
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1197 # size can be printed here because .stat is
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1198 # available
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1199 #
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1200 self._writer.write_size(opath, fso.stat.st_size)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1201 else:
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1202 try:
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1203 dgst = digest.compute_digest_file(
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1204 self._algorithm[0],
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1205 fso.path,
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1206 use_mmap=self._use_mmap)
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1207 except OSError as e:
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1208 dir_tainted = True
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1209 self._writer.write_error(
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1210 util.interpolate_bytes(
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1211 b"`%s': errno %d: %s",
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1212 opath,
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1213 e.errno,
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1214 util.b(e.strerror, "utf-8")))
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1215 sz = (fso.stat.st_size if self._print_size
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1216 else None)
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1217 self._writer.write_file_digest(
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	1218 self._algorithm[1], opath, None, sz)
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1219 else:
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1220 dir_dgst.update(util.interpolate_bytes(
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1221 b"%d:%s,%d:%s,",
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1222 len(self._algorithm[1]),
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1223 util.b(self._algorithm[1]),
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1224 len(dgst),
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1225 dgst))
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1226 sz = (fso.stat.st_size if self._print_size
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1227 else None)
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1228 self._writer.write_file_digest(
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	1229 self._algorithm[1], opath, dgst, sz)
267 b9aa65a30b4c treesum: optimize the use of flush() somewhat Franz Glasner <fzglas.hg@dom66.de> parents: 266 diff changeset	1230 self._writer.flush()
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1231 opath = join_output_path(top, None)
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1232 if opath:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1233 if self._utf8_mode:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1234 opath = walk.WalkDirEntry.alt_u8(opath)
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1235 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1236 opath = walk.WalkDirEntry.alt_fs(opath)
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1237 if dir_tainted:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1238 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1239 # IMPORTANT: Print errors BEFORE the associated digest or size
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1240 # line. Otherwise the "info" command has a problem.
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1241 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1242 self._writer.write_error(b"directory is tainted")
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1243 logging.error("Directory has problems: %r", opath)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1244 if self._size_only:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1245 self._writer.write_size(opath, dir_size)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1246 else:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1247 sz = dir_size if self._print_size else None
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1248 self._writer.write_file_digest(
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	1249 self._algorithm[1], opath, dir_dgst.digest(), sz)
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1250 self._writer.flush()
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1251 return (0, self._algorithm[1], dir_dgst.digest(), dir_size)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1252
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1253
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1254 def join_output_path(top, name):
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1255 if name is None:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1256 # a path for a directory is to be computed
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1257 if top:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1258 if isinstance(top[0], bytes):
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1259 return b"/".join(top) + b"/"
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1260 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1261 return u"/".join(top) + u"/"
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1262 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1263 return b""
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1264 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1265 # a path for a normal file is to be computed
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1266 if top:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1267 if isinstance(name, bytes):
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1268 return b"/".join(top) + b"/" + name
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1269 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1270 return u"/".join(top) + u"/" + name
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1271 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1272 return name
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1273
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1274
158 d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	1275 def normalized_compatible_mode_str(mode):
157 27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	1276 # XXX FIXME: Windows and "executable"
27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	1277 modebits = stat.S_IMODE(mode)
27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	1278 modestr = "%o" % (modebits,)
27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	1279 if not modestr.startswith("0"):
27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	1280 modestr = "0" + modestr
27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	1281 return modestr
27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	1282
27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	1283
158 d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	1284 def normalized_mode_str(mode):
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	1285 modestr = "%o" % (mode,)
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	1286 if not modestr.startswith("0"):
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	1287 modestr = "0" + modestr
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	1288 return modestr
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	1289
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	1290
325 cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1291 class WriterBase(object):
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1292
336 0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1293 """(Abstract) base class for all treesum digest file writers.
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1294
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1295 Wraps an output file pointer for a binary file.
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1296
325 cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1297 Provides low-level methods to write data lines. These methods must be
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1298 used if the CRC is to be updated also.
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1299
333 2a244a05bbb5 treesum: FIX: docs Franz Glasner <fzglas.hg@dom66.de> parents: 332 diff changeset	1300 Also holds some very common attributes that control some aspects
2a244a05bbb5 treesum: FIX: docs Franz Glasner <fzglas.hg@dom66.de> parents: 332 diff changeset	1301 of the output format (e.g. `.LS`, `.use_base64`).
2a244a05bbb5 treesum: FIX: docs Franz Glasner <fzglas.hg@dom66.de> parents: 332 diff changeset	1302
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1303 Also holds the current CRC for a block.
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1304
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1305 """
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1306
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1307 LS = util.b(os.linesep)
332 5b98810fe367 treesum: attribute comment for "LS" Franz Glasner <fzglas.hg@dom66.de> parents: 331 diff changeset	1308 """Because we write the output as binary files we need the official line
5b98810fe367 treesum: attribute comment for "LS" Franz Glasner <fzglas.hg@dom66.de> parents: 331 diff changeset	1309 separator for you OS as bytes"""
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1310
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1311 DEFAULT_GROUPING_SEPARATOR = ""
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1312 """Disable the thousands separator in case no subclass redefines it"""
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1313
336 0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1314 def __init__(self, outfp, size_only=False, print_size=False,
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1315 use_base64=False, grouping_separator=None,
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1316 size_column_width=None):
336 0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1317 # Poor man's abstract abstract class implemenetation
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1318 assert self.__class__ is not WriterBase
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1319
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1320 self._outfp = outfp
336 0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1321 self.size_only = size_only
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1322 self.print_size = print_size
331 9ee84624587f treesum: move the handling of base64 digest output into the writers completely Franz Glasner <fzglas.hg@dom66.de> parents: 330 diff changeset	1323 self.use_base64 = use_base64
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1324 self.grouping_separator = (grouping_separator
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1325 if grouping_separator is not None
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1326 else self.DEFAULT_GROUPING_SEPARATOR)
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1327 self.size_column_width = size_column_width or 0
325 cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1328 self.reset_crc()
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1329
336 0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1330 def write_size(self, filename, sz):
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1331 """
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1332
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1333 If `sz` is `None` then this is signals an error because a size is
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1334 required.
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1335
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1336 If the size should not be printed on purpose the `size` should be
e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1337 as negative number.
e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1338
336 0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1339 """
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1340 raise NotImplementedError("write_size")
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1341
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	1342 def write_file_digest(self, algorithm, filename, digest, size):
336 0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1343 """
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1344
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1345 If `size` is `None` and the output of a size is required then this
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1346 is an error signal.
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1347
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1348 If the size should not be printed on purpose the `size` should be
e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1349 as negative number.
e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1350
e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1351 If `digest` is `None` is an error signal.
e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1352
336 0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1353 """
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1354 raise NotImplementedError("write_file_digest")
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1355
325 cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1356 @property
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1357 def crc(self):
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1358 return self._crc
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1359
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1360 def reset_crc(self):
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1361 self._crc = crc32()
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1362
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1363 def writeln(self, line):
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1364 """Write the bytes `line` into the output file and update the CRC
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1365 accordingly.
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1366
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1367 :param bytes line: The line to write to (without line ending)
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1368
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1369 """
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1370 self.write(line)
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1371 self.write(self.LS)
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1372
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1373 def write(self, data):
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1374 """Write `data` into the output file and update the CRC accordingly.
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1375
325 cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1376 :param bytes data: The data to write to and to update the CRC with
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1377
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1378 """
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1379 if data:
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1380 self._outfp.write(data)
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1381 self._crc.update(data)
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1382
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1383 def flush(self):
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1384 self._outfp.flush()
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1385
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1386
326 a464f36fffcb treesum: Rename the current TreesumWriter to TaggedTreesumWriter. Franz Glasner <fzglas.hg@dom66.de> parents: 325 diff changeset	1387 class TaggedTreesumWriter(WriterBase):
325 cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1388
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1389 """Writer to write treesum digest files in a format similar to BSD
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1390 digest files.
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1391
333 2a244a05bbb5 treesum: FIX: docs Franz Glasner <fzglas.hg@dom66.de> parents: 332 diff changeset	1392 Provides high-level methods to write data lines.
325 cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1393
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1394 """
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1395
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1396 DEFAULT_GROUPING_SEPARATOR = '_'
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1397 """The default thousands separator"""
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1398
325 cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1399 def __init__(self, outfp, **kwds):
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1400 # No alignment for the size here
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1401 kwds["size_column_width"] = 0
331 9ee84624587f treesum: move the handling of base64 digest output into the writers completely Franz Glasner <fzglas.hg@dom66.de> parents: 330 diff changeset	1402 super(TaggedTreesumWriter, self).__init__(outfp, **kwds)
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1403
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1404 def start(self, version):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1405 """Begin a new block, reset the current CRC and write the VERSION
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1406 tag.
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1407
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1408 """
325 cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1409 self.reset_crc()
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1410 self.write(b"VERSION = ")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1411 self.writeln(util.b(version))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1412
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1413 def write_comment(self, comment):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1414 self.write(b"COMMENT (")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1415 self.write(util.b(comment, "utf-8"))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1416 self.writeln(b")")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1417
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1418 def write_generator(self, generator):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1419 self.write(b"GENERATOR (")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1420 self.write(util.b(generator, "utf-8"))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1421 self.writeln(b")")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1422
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1423 def write_error(self, error):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1424 self.write(b"ERROR (")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1425 self.write(util.b(error, "utf-8"))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1426 self.writeln(b")")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1427
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1428 def write_fsencoding(self, encoding):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1429 self.write(b"FSENCODING = ")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1430 self.writeln(util.b(encoding))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1431
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1432 def write_fnmatch_pattern(self, action, kind, pattern):
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1433 self.write(b"FNMATCH (")
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1434 self.write(util.b(action))
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1435 self.write(b": ")
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1436 self.write(util.b(kind))
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1437 self.write(b":")
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1438 self.write(util.b(pattern, "utf-8"))
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1439 self.writeln(b")")
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1440
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1441 def write_flags(self, flags):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1442 self.write(b"FLAGS = ")
262 c3d6599c1b5e FIX: treesum: bytearray needs not to be handled Franz Glasner <fzglas.hg@dom66.de> parents: 261 diff changeset	1443 if isinstance(flags, (str, bytes)):
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1444 self.writeln(util.b(flags))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1445 else:
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1446 flags.sort()
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1447 self.writeln(util.b(",".join(flags)))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1448
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1449 def write_timestamp(self, ts):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1450 self.write(b"TIMESTAMP = ")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1451 self.writeln(util.b(str(ts)))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1452
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1453 def write_isotimestamp(self, ts):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1454 self.write(b"ISOTIMESTAMP = ")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1455 self.writeln(util.b(ts))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1456
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1457 def write_root(self, root):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1458 assert isinstance(root, bytes)
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1459 self.write(b"ROOT (")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1460 self.write(root)
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1461 self.writeln(b")")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1462
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1463 def write_size(self, filename, sz):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1464 assert isinstance(filename, bytes)
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1465 self.write(b"SIZE (")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1466 self.write(filename)
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1467 self.write(b")")
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1468 if sz is not None:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1469 self.write(b" = ")
339 9c7a03199092 treesum: Use the grouping separator (aka thousands separator) when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 338 diff changeset	1470 self.write(
9c7a03199092 treesum: Use the grouping separator (aka thousands separator) when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 338 diff changeset	1471 b"" if sz < 0
9c7a03199092 treesum: Use the grouping separator (aka thousands separator) when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 338 diff changeset	1472 else util.b(format(sz, ',').replace(
9c7a03199092 treesum: Use the grouping separator (aka thousands separator) when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 338 diff changeset	1473 ',', self.grouping_separator)))
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1474 self.writeln(b"")
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1475
317 fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1476 def write_accept_treesum_file(self, filename):
fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1477 assert isinstance(filename, bytes)
fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1478 self.write(b"ACCEPT-TREESUM (")
fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1479 self.write(filename)
fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1480 self.writeln(b")")
fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1481
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	1482 def write_file_digest(self, algorithm, filename, digest, size):
314 1fbe9904b188 treesum: assert also in .write_accept_treesum() that the given filename is of bytes type Franz Glasner <fzglas.hg@dom66.de> parents: 313 diff changeset	1483 assert isinstance(filename, bytes)
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1484 if digest is not None:
341 728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1485 if digest != b"":
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1486 digest = (base64.b64encode(digest)
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1487 if self.use_base64
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1488 else binascii.hexlify(digest))
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1489 self.write(util.b(algorithm))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1490 self.write(b" (")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1491 self.write(filename)
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1492 self.write(b")")
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1493 if digest is not None or size is not None:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1494 self.write(b" = ")
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1495 if digest is not None:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1496 self.write(digest)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1497 if size is not None:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1498 self.write(b",")
339 9c7a03199092 treesum: Use the grouping separator (aka thousands separator) when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 338 diff changeset	1499 self.write(
9c7a03199092 treesum: Use the grouping separator (aka thousands separator) when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 338 diff changeset	1500 b"" if size < 0
9c7a03199092 treesum: Use the grouping separator (aka thousands separator) when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 338 diff changeset	1501 else util.b(format(size, ',').replace(
9c7a03199092 treesum: Use the grouping separator (aka thousands separator) when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 338 diff changeset	1502 ',', self.grouping_separator)))
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1503 self.writeln(b"")
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1504
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1505 def finish(self):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1506 """Finish a block and write the current CRC"""
325 cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1507 crc = self.crc.hexdigest()
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1508 self.write(b"CRC32 = ")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1509 self.writeln(util.b(crc))
267 b9aa65a30b4c treesum: optimize the use of flush() somewhat Franz Glasner <fzglas.hg@dom66.de> parents: 266 diff changeset	1510 self.flush()
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1511
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	1512
330 61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1513 class TabularTreesumWriter(WriterBase):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1514
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1515 """Writer to write treesum digest files in a format similar to tabular
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1516 GNU digest files.
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1517
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1518 Provides high-level methods to write data lines.
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1519
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1520 """
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1521
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1522 DEFAULT_GROUPING_SEPARATOR = '.'
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1523 """The default thousands separator"""
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1524
330 61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1525 def __init__(self, outfp, **kwds):
331 9ee84624587f treesum: move the handling of base64 digest output into the writers completely Franz Glasner <fzglas.hg@dom66.de> parents: 330 diff changeset	1526 super(TabularTreesumWriter, self).__init__(outfp, **kwds)
341 728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1527 # Prepare some format strings for performance reasons
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1528 if self.size_column_width > 0:
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1529 self._formatstring_size = '>' + str(self.size_column_width)
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1530 self._errorstring_size = b'?' * self.size_column_width
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1531 self._emptystring_size = b' ' * self.size_column_width
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1532 else:
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1533 self._formatstring_size = ">"
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1534 self._errorstring_size = b"?????"
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1535 self._emptystring_size = b''
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1536 if self.grouping_separator:
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1537 self._formatstring_size += ','
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1538 self._formatstring_size += 'd'
330 61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1539
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1540 def start(self, version):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1541 """Begin a new block, reset the current CRC and write the VERSION
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1542 tag.
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1543
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1544 """
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1545 self.reset_crc()
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1546 self.write(b"VERSION\t")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1547 self.writeln(util.b(version))
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1548
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1549 def write_comment(self, comment):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1550 self.write(b"COMMENT\t")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1551 self.writeln(util.b(comment, "utf-8"))
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1552
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1553 def write_generator(self, generator):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1554 self.write(b"GENERATOR\t")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1555 self.writeln(util.b(generator, "utf-8"))
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1556
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1557 def write_error(self, error):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1558 self.write(b"ERROR\t")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1559 self.writeln(util.b(error, "utf-8"))
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1560
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1561 def write_fsencoding(self, encoding):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1562 self.write(b"FSENCODING\t")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1563 self.writeln(util.b(encoding))
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1564
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1565 def write_fnmatch_pattern(self, action, kind, pattern):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1566 self.write(b"FNMATCH\t")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1567 self.write(util.b(action))
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1568 self.write(b": ")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1569 self.write(util.b(kind))
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1570 self.write(b":")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1571 self.writeln(util.b(pattern, "utf-8"))
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1572
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1573 def write_flags(self, flags):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1574 self.write(b"FLAGS\t")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1575 if isinstance(flags, (str, bytes)):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1576 self.writeln(util.b(flags))
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1577 else:
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1578 flags.sort()
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1579 self.writeln(util.b(",".join(flags)))
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1580
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1581 def write_timestamp(self, ts):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1582 self.write(b"TIMESTAMP\t")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1583 self.writeln(util.b(str(ts)))
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1584
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1585 def write_isotimestamp(self, ts):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1586 self.write(b"ISOTIMESTAMP\t")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1587 self.writeln(util.b(ts))
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1588
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1589 def write_root(self, root):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1590 assert isinstance(root, bytes)
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1591 self.write(b"ROOT\t")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1592 self.writeln(root)
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1593
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1594 def write_size(self, filename, sz):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1595 assert isinstance(filename, bytes)
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1596 if sz is not None:
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1597 if sz >= 0:
341 728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1598 self.write(util.b(format(
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1599 sz, self._formatstring_size).replace(
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1600 ',', self.grouping_separator)))
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1601 else:
341 728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1602 self.write(self._emptystring_size)
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1603 else:
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1604 self.write(self._errorstring_size)
330 61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1605 self.write(b"\t")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1606 self.writeln(filename)
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1607
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1608 def write_accept_treesum_file(self, filename):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1609 assert isinstance(filename, bytes)
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1610 self.write(b"ACCEPT-TREESUM\t")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1611 self.writeln(filename)
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1612
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	1613 def write_file_digest(self, algorithm, filename, digest, size):
330 61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1614 assert isinstance(filename, bytes)
341 728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1615 if digest is not None and digest != b"":
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1616 digest = (base64.b64encode(digest) if self.use_base64
330 61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1617 else binascii.hexlify(digest))
341 728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1618 else:
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1619 #
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1620 # Compute an error digest string with the "correct" length for
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1621 # given algorithm
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1622 #
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1623 nulldigest = b'\0' * util.algotag2digest_size(algorithm)
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1624 dsz = len(base64.b64encode(nulldigest) if self.use_base64
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1625 else binascii.hexlify(nulldigest))
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1626 if digest is None:
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1627 digest = b'?' * dsz
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1628 else:
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1629 digest = b' ' * dsz
330 61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1630 self.write(util.b(algorithm))
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1631 self.write(b":")
341 728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1632 self.write(digest)
330 61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1633 self.write(b"\t")
341 728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1634 if self.print_size:
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1635 if size is not None:
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1636 if size >= 0:
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1637 self.write(util.b(format(
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1638 size, self._formatstring_size).replace(
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1639 ',', self.grouping_separator)))
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1640 else:
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1641 self.write(self._emptystring_size)
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1642 else:
341 728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1643 self.write(self._errorstring_size)
330 61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1644 self.write(b"\t")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1645 self.writeln(filename)
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1646
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1647 def finish(self):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1648 """Finish a block and write the current CRC"""
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1649 crc = self.crc.hexdigest()
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1650 self.write(b"CRC32\t")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1651 self.writeln(util.b(crc))
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1652 self.flush()
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1653
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1654
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1655 class TreesumReader(object):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1656
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1657 """Reader to read and/or verify treesum digest files.
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1658
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1659 Supports the iterator and context manager protocol.
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1660
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1661 """
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1662
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1663 PATTERN0 = re.compile(br"\A[ \t]*\r?\n\Z") # empty lines
198 c1e875ba4bdc Put the effective filesystem encoding into the treesum digest file using FSENCODING = <encoding> Franz Glasner <fzglas.hg@dom66.de> parents: 197 diff changeset	1664 PATTERN1 = re.compile(br"\A(VERSION\|FSENCODING\|FLAGS\|TIMESTAMP\|ISOTIMESTAMP\|CRC32)[ \t]=[ \t]([^ \t]+)[ \t]*\r?\n\Z") # noqa: E501 line too long
317 fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1665 PATTERN2 = re.compile(br"\A(ROOT\|COMMENT\|ERROR\|GENERATOR\|FNMATCH\|ACCEPT-TREESUM)[ \t]\((.)\)[ \t]*\r?\n\Z") # noqa: E501 line too long
340 cfa544fbb9f9 treesum: allow parsing of files again where sizes are printed with grouping with standard separators out of "., '_" Franz Glasner <fzglas.hg@dom66.de> parents: 339 diff changeset	1666 PATTERN3 = re.compile(br"\ASIZE[ \t]\((.)\)([ \t]=[ \t]([0-9., '_][0-9]))?[ \t]\r?\n\Z") # noqa: E501 line too long
cfa544fbb9f9 treesum: allow parsing of files again where sizes are printed with grouping with standard separators out of "., '_" Franz Glasner <fzglas.hg@dom66.de> parents: 339 diff changeset	1667 PATTERN4 = re.compile(br"\A([A-Za-z0-9_-]+)[ \t]\((.)\)([ \t]=[ \t]([A-Za-z0-9=+/]+)?(,([0-9., '_][0-9])?)?)?[ \t]\r?\n\Z") # noqa: E501 line too long
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1668
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1669 def __init__(self, _fp, _filename, _own_fp):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1670 self._fp = _fp
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1671 self._own_fp = _own_fp
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1672 self._filename = _filename
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1673 self._line_no = 0
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1674 self._reset_crc()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1675 self._expect_crc = None # NOTE: tristate: None is different from False
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1676 self._current_algo_name = self._current_algo_digest_size = None
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1677
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1678 @classmethod
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1679 def from_path(cls_, path):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1680 """Open file at `path` and return a reader that owns the file object"""
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1681 return cls_(open(path, "rb"), path, True)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1682
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1683 @classmethod
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1684 def from_binary_buffer(cls_, binary_fp, filename):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1685 return cls_(binary_fp, filename, False)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1686
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1687 def __enter__(self):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1688 return self
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1689
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1690 def __exit__(self, *args):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1691 self.close()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1692
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1693 def close(self):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1694 if self._fp is not None:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1695 try:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1696 if self._own_fp:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1697 self._fp.close()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1698 finally:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1699 self._fp = None
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1700
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1701 def __iter__(self):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1702 return self
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1703
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1704 def __next__(self):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1705 rec = self.read_record()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1706 if rec is None:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1707 raise StopIteration()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1708 return rec
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1709
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1710 if util.PY2:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1711 next = __next__
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1712
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1713 def all_records(self):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1714 """Iterator over all remaining records"""
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1715 while True:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1716 rec = self.read_record()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1717 if rec is None:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1718 return
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1719 yield rec
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1720
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1721 def read_record(self):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1722 """Read and parse the "next" line.
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1723
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1724 :returns: `None` at EOF or the parsed contents of the line
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1725 :rtype: tuple or None
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1726
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1727 """
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1728 # Loop to skip empty lines
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1729 while True:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1730 line = self._get_next_line()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1731 if not line:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1732 #
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1733 # Skip for empty files at the very beginning.
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1734 # Check only after the first VERSION line.
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1735 #
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1736 if self._expect_crc is not None:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1737 if self._expect_crc:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1738 logging.warning("CRC32 is missing at EOF")
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1739 return None
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1740 if not self.PATTERN0.search(line):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1741 break
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1742 self._update_crc(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1743 #
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1744 # At the beginning transparently skip an eventually embedded signify
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1745 # signature
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1746 #
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1747 if self._line_no == 1:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1748 if line.startswith(b"untrusted comment: "):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1749 line = self._get_next_line()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1750 if not line.endswith(b"\n"):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1751 raise binascii.Error("No valid signify signature value")
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1752 # Try to decode for an early error check
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1753 base64.b64decode(line[:-1])
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1754 mo = self.PATTERN1.search(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1755 if mo:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1756 if mo.group(1) == b"VERSION":
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1757 if self._expect_crc:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1758 logging.warning("CRC32 missing before line %d",
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1759 self._line_no)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1760 self._reset_crc()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1761 self._expect_crc = True
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1762 self._update_crc(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1763 return ("VERSION", util.n(mo.group(2)))
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1764 if mo.group(1) == b"CRC32":
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1765 # TODO: check
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1766 if self._expect_crc is None:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1767 logging.warning("Lone CRC32 before VERSION in line %d",
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1768 self._line_no)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1769 else:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1770 if self._expect_crc:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1771 if (self._hex_crc()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1772 != mo.group(2).decode("latin1").upper()):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1773 logging.warning(
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1774 "CRC32 mismatch in line %d:"
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1775 " expected: %s, given: %s",
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1776 self._line_no,
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1777 self._hex_crc(),
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1778 mo.group(2).decode("latin1").upper())
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1779 else:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1780 logging.warning("CRC32 before VERSION in line %d",
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1781 self._line_no)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1782 # Do not update the CRC here but reset the state
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1783 self._expect_crc = False
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1784 return ("CRC32", util.n(mo.group(2)))
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1785 else:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1786 self._update_crc(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1787 return (util.n(mo.group(1)), util.n(mo.group(2)))
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1788 else:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1789 mo = self.PATTERN2.search(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1790 if mo:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1791 self._update_crc(line)
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1792 if mo.group(1) in (b"COMMENT", b"ERROR", b"GENERATOR",
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1793 b"FNMATCH"):
316 3f1f1c2e9e5f treesum: FIX: returned tag should be a native string Franz Glasner <fzglas.hg@dom66.de> parents: 315 diff changeset	1794 return (util.n(mo.group(1)), util.u(mo.group(2), "utf-8"))
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1795 elif mo.group(1) == b"ROOT":
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1796 return ("ROOT", mo.group(2))
317 fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1797 elif mo.group(1) == b"ACCEPT-TREESUM":
fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1798 return ("ACCEPT-TREESUM", mo.group(2))
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1799 assert False, line
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1800 else:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1801 mo = self.PATTERN3.search(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1802 if mo:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1803 self._update_crc(line)
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1804 if mo.group(2):
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1805 return ("SIZE", mo.group(1),
340 cfa544fbb9f9 treesum: allow parsing of files again where sizes are printed with grouping with standard separators out of "., '_" Franz Glasner <fzglas.hg@dom66.de> parents: 339 diff changeset	1806 int(util.n(mo.group(3))
cfa544fbb9f9 treesum: allow parsing of files again where sizes are printed with grouping with standard separators out of "., '_" Franz Glasner <fzglas.hg@dom66.de> parents: 339 diff changeset	1807 .translate(None, "., '_"),
cfa544fbb9f9 treesum: allow parsing of files again where sizes are printed with grouping with standard separators out of "., '_" Franz Glasner <fzglas.hg@dom66.de> parents: 339 diff changeset	1808 10))
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1809 else:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1810 return ("SIZE", mo.group(1), None)
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1811 else:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1812 mo = self.PATTERN4.search(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1813 if mo:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1814 self._update_crc(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1815 algo_name = util.n(mo.group(1))
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1816 if mo.group(3):
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1817 if mo.group(4):
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1818 if (len(mo.group(4)) ==
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1819 2 * self._get_digest_size(algo_name)):
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1820 # hex
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1821 digest = binascii.unhexlify(mo.group(4))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1822 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1823 # base64
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1824 digest = base64.b64decode(mo.group(4))
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1825 else:
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1826 digest = None
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1827 if mo.group(5):
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1828 if mo.group(6):
340 cfa544fbb9f9 treesum: allow parsing of files again where sizes are printed with grouping with standard separators out of "., '_" Franz Glasner <fzglas.hg@dom66.de> parents: 339 diff changeset	1829 size = int(util.n(mo.group(6))
cfa544fbb9f9 treesum: allow parsing of files again where sizes are printed with grouping with standard separators out of "., '_" Franz Glasner <fzglas.hg@dom66.de> parents: 339 diff changeset	1830 .translate(None, "., '_"),
cfa544fbb9f9 treesum: allow parsing of files again where sizes are printed with grouping with standard separators out of "., '_" Franz Glasner <fzglas.hg@dom66.de> parents: 339 diff changeset	1831 10)
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1832 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1833 size = None
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1834 else:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1835 size = None
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1836 return (algo_name, mo.group(2), digest, size)
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1837 else:
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1838 return (algo_name, mo.group(2), None, None)
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1839 else:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1840 assert False, line
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1841 return line
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1842
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1843 def _get_next_line(self):
189 959c6d37b014 Extend the max line length to read to 4096, which is something along PATH_MAX onn Linux Franz Glasner <fzglas.hg@dom66.de> parents: 188 diff changeset	1844 line = self._fp.readline(4096) # along PATH_MAX on Linux
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1845 if line:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1846 self._line_no += 1
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1847 return line
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1848
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1849 def _reset_crc(self):
260 07a0bc723139 treesum: Implement the CRC-32 using the new util.crc32 module Franz Glasner <fzglas.hg@dom66.de> parents: 223 diff changeset	1850 self._crc32 = crc32()
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1851
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1852 def _update_crc(self, data):
260 07a0bc723139 treesum: Implement the CRC-32 using the new util.crc32 module Franz Glasner <fzglas.hg@dom66.de> parents: 223 diff changeset	1853 self._crc32.update(data)
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1854
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1855 def _hex_crc(self):
260 07a0bc723139 treesum: Implement the CRC-32 using the new util.crc32 module Franz Glasner <fzglas.hg@dom66.de> parents: 223 diff changeset	1856 return self._crc32.hexdigest()
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1857
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1858 def _get_digest_size(self, algo_name):
307 64df94bf4659 treesum: Build a little static database of digest sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 306 diff changeset	1859 """Get the `digest_size` from algorithm specifier `algo_name`.
64df94bf4659 treesum: Build a little static database of digest sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 306 diff changeset	1860
64df94bf4659 treesum: Build a little static database of digest sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 306 diff changeset	1861 Cache this on the assumption, that algorithms do not change very
64df94bf4659 treesum: Build a little static database of digest sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 306 diff changeset	1862 often. Do this because the `digest_size` can only be given by a
64df94bf4659 treesum: Build a little static database of digest sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 306 diff changeset	1863 digest instance.
64df94bf4659 treesum: Build a little static database of digest sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 306 diff changeset	1864
64df94bf4659 treesum: Build a little static database of digest sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 306 diff changeset	1865 """
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1866 if self._current_algo_name == algo_name:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1867 return self._current_algo_digest_size
307 64df94bf4659 treesum: Build a little static database of digest sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 306 diff changeset	1868 sz = util.algotag2digest_size(algo_name)
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1869 self._current_algo_name = algo_name
307 64df94bf4659 treesum: Build a little static database of digest sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 306 diff changeset	1870 self._current_algo_digest_size = sz
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1871 return self._current_algo_digest_size
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1872
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1873
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1874 def print_treesum_digestfile_infos(opts):
306 ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1875 get_infos_from_digestfile(
ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1876 opts.digest_files,
ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1877 print_block_data,
ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1878 opts.print_only_last_block)
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1879
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1880
306 ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1881 def get_infos_from_digestfile(digest_files, block_handler,
ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1882 only_last_block=True):
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1883 for fn in digest_files:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1884 if fn == "-":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1885 if util.PY2:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1886 reader = TreesumReader.from_binary_buffer(sys.stdin)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1887 else:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1888 reader = TreesumReader.from_binary_buffer(sys.stdin.buffer)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1889 else:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1890 reader = TreesumReader.from_path(fn)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1891
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1892 with reader:
208 85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1893 root = generator = flags = fsencoding = algorithm = digest \
85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1894 = size = None
219 19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1895 errors = set()
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1896 comments = []
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1897 fnmatch_filters = []
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1898 in_block = False
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1899 block_no = 0
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1900 for record in reader:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1901 if record[0] == "VERSION":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1902 assert record[1] == "1"
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1903 # start a new block
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1904 in_block = True
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1905 block_no += 1
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1906 root = flags = algorithm = digest = size = None
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1907 comments = []
208 85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1908 elif record[0] == "GENERATOR":
85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1909 generator = record[1]
85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1910 elif record[0] == "FSENCODING":
85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1911 fsencoding = record[1]
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1912 elif record[0] == "FLAGS":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1913 flags = record[1]
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1914 elif record[0] == "ROOT":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1915 root = record[1]
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1916 elif record[0] == "COMMENT":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1917 comments.append(record[1])
219 19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1918 elif record[0] == "ERROR":
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1919 errors.add(record[1])
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1920 elif record[0] == "FNMATCH":
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1921 fnmatch_filters.append(record[1])
208 85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1922 elif record[0] in ("TIMESTAMP", "ISOTIMESTAMP"):
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1923 pass
317 fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1924 elif record[0] == "ACCEPT-TREESUM":
fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1925 pass
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1926 elif record[0] == "CRC32":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1927 pass
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1928 # in_block = False
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1929 else:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1930 if not in_block:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1931 continue
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1932 # digest line or size line
193 fb36e71f6ba8 Change: path indicators for symlinks: ./@ -> ./@/ and /./@ -> /./@/ Franz Glasner <fzglas.hg@dom66.de> parents: 191 diff changeset	1933 if not record[1] or record[1] == b"./@/":
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1934 if record[0] == "SIZE":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1935 algorithm = "SIZE"
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1936 digest = None
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1937 size = record[2]
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1938 else:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1939 algorithm = record[0]
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1940 digest = record[2]
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1941 size = record[3]
306 ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1942 if not only_last_block:
ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1943 block_handler(
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1944 block_no,
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1945 root, generator, fsencoding, flags,
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1946 fnmatch_filters,
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1947 comments, errors, algorithm, digest, size)
219 19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1948 root = generator = flags = fsencoding = algorithm \
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1949 = digest = size = None
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1950 errors = set()
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1951 comments = []
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1952 in_block = False
306 ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1953 if only_last_block:
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1954 if not in_block:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1955 if digest is not None or size is not None:
306 ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1956 block_handler(
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1957 block_no,
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1958 root, generator, fsencoding, flags, fnmatch_filters,
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1959 comments, errors, algorithm, digest, size)
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1960 else:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1961 logging.warning("missing block end")
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1962
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1963
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1964 def print_block_data(block_no, tag, generator, fsencoding, flags,
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1965 fnmatch_filters, comments, errors,
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1966 algorithm, digest, size):
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1967 digeststr = util.n(binascii.hexlify(digest)) if digest else "<no digest>"
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1968 sizestr = str(size) if size is not None else "<no size>"
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1969 print("BLOCK No %d:" % (block_no,))
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1970 print(" Tag:", tag)
208 85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1971 print(" FS-Encoding:", fsencoding)
85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1972 if generator:
85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1973 print(" Generator:", generator)
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1974 print(" Flags:", flags if flags else "<none>")
220 8db78850d800 "treesum info": print comments only if not empty Franz Glasner <fzglas.hg@dom66.de> parents: 219 diff changeset	1975 if comments:
8db78850d800 "treesum info": print comments only if not empty Franz Glasner <fzglas.hg@dom66.de> parents: 219 diff changeset	1976 print(" Comments:", comments)
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1977 if fnmatch_filters:
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1978 for f in fnmatch_filters:
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1979 print(" FNMatch:", f)
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1980 print(" Algorithm:", algorithm)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1981 if algorithm != "SIZE":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1982 print(" Digest:", digeststr)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1983 print(" Size:", sizestr)
303 73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	1984 if errors:
73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	1985 errorlist = list(errors)
73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	1986 errorlist.sort()
73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	1987 for idx, err in enumerate(errorlist):
73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	1988 if idx == 0:
73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	1989 print(" Errors:", err)
73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	1990 else:
73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	1991 print(" ", err)
73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	1992 else:
73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	1993 print(" Errors: <none>")
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1994
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1995
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1996 class DigestSizeCollector(object):
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1997
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1998 def __init__(self):
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1999 self._algorithm = self._digest = self._size = None
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2000
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2001 def __call__(self, block_no, tag, generator, fsencoding, flags,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2002 fnmatch_filters, comments, errors,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2003 algorithm, digest, size):
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2004 self._algorithm = algorithm
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2005 self._digest = digest
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2006 self._size = size
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2007
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2008 @property
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2009 def algorithm(self):
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2010 return self._algorithm
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2011
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2012 @property
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2013 def digest(self):
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2014 return self._digest
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2015
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2016 @property
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2017 def size(self):
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2018 return self._size
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2019
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2020 def collect_from_file(self, digest_file):
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2021 get_infos_from_digestfile([digest_file], self, True)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2022
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2023
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	2024 if __name__ == "__main__":
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	2025 sys.exit(main())

Mercurial > hgrepos > Python > apps > py-cutils

annotate cutils/treesum.py @ 347:3f4840eeb0f4