Python/apps/py-cutils: cutils/treesum.py annotate

annotate cutils/treesum.py @ 341:728ad9c639f2

treesum: right-align the output of size when using the tabular style

author	Franz Glasner <fzglas.hg@dom66.de>
date	Mon, 31 Mar 2025 15:24:48 +0200
parents	cfa544fbb9f9
children	0a58948df713

rev	line source
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	1 # -- coding: utf-8 --
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	2 # :-
323 48430941c18c Adopt copyright and license wordings from https://reuse.software/faq/. Franz Glasner <fzglas.hg@dom66.de> parents: 322 diff changeset	3 # SPDX-FileCopyrightText: © 2025 Franz Glasner
48430941c18c Adopt copyright and license wordings from https://reuse.software/faq/. Franz Glasner <fzglas.hg@dom66.de> parents: 322 diff changeset	4 # SPDX-License-Identifier: BSD-3-Clause
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	5 # :-
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	6 r"""Generate and verify checksums for directory trees.
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	7
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	8 """
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	9
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	10 from __future__ import print_function, absolute_import
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	11
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	12
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	13 __all__ = []
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	14
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	15
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	16 import argparse
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	17 import base64
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	18 import binascii
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	19 import collections
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	20 import datetime
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	21 import errno
176 7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	22 import logging
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	23 import os
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	24 import re
157 27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	25 import stat
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	26 import sys
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	27 import time
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	28
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	29 from . import (__version__, __revision__)
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	30 from . import util
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	31 from .util import cm
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	32 from .util import digest
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	33 from .util import fnmatch
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	34 from .util import walk
260 07a0bc723139 treesum: Implement the CRC-32 using the new util.crc32 module Franz Glasner <fzglas.hg@dom66.de> parents: 223 diff changeset	35 from .util.crc32 import crc32
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	36
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	37
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	38 def main(argv=None):
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	39
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	40 def _populate_generate_arguments(gp):
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	41 """Use to populate command aliases.
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	42
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	43 This is because :class:`argparse.ArgumentParser` does not
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	44 support them for all supported Python versions.
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	45
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	46 """
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	47 gp.add_argument(
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	48 "--accept-treesum", "-A", action=PatternMatchAction,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	49 kind="accept-treesum",
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	50 dest="fnmatch_filters", metavar="PATTERN",
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	51 help="""Accept an existing treesum file PATTERN for a directory
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	52 checksum.
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	53 Implicitly this also acts as `--exclude' option.
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	54 Can be given more than once.""")
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	55 gp.add_argument(
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	56 "--algorithm", "-a", action="store", type=util.argv2algo,
172 804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	57 help="1 (aka sha1), 224, 256 (aka sha256), 384, 512 (aka sha512), "
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	58 "3 (alias for sha3-512), 3-224, 3-256, 3-384, 3-512, "
172 804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	59 "blake2b, blake2b-256, blake2s, "
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	60 "blake2 (alias for blake2b), "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	61 "blake2-256 (alias for blake2b-256), "
172 804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	62 "md5. "
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	63 "The default depends on availability in hashlib: "
804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	64 "blake2b-256, sha256 or sha1.")
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	65 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	66 "--append-output", action="store_true", dest="append_output",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	67 help="Append to the output file instead of overwriting it.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	68 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	69 "--base64", action="store_true",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	70 help="Output checksums in base64 notation, not hexadecimal "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	71 "(OpenBSD).")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	72 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	73 "--comment", action="append", default=[],
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	74 help="Put given comment COMMENT into the output as \"COMMENT\". "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	75 "Can be given more than once.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	76 gp.add_argument(
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	77 "--exclude", "-X", action=PatternMatchAction, kind="exclude",
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	78 dest="fnmatch_filters", metavar="PATTERN",
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	79 help="""Exclude names matching the given PATTERN.
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	80 For help on PATTERN use \"help patterns\".
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	81 Can be given more than once.""")
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	82 gp.add_argument(
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	83 "--follow-directory-symlinks", "-l", action=SymlinkAction,
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	84 const="follow-directory-symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	85 default=FollowSymlinkConfig(False, False, True),
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	86 dest="follow_symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	87 help="""Follow symbolic links to directories when walking a
271 6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	88 directory tree. Augments --physical, --half and -p.""")
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	89 gp.add_argument(
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	90 "--follow-file-symlinks", action=SymlinkAction,
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	91 const="follow-file-symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	92 dest="follow_symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	93 help="""Follow symbolic links to files when walking a
271 6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	94 directory tree. Augments --physical and --half.""")
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	95 gp.add_argument(
158 d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	96 "--full-mode", action="store_true", dest="metadata_full_mode",
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	97 help="Consider all mode bits as returned from stat(2) when "
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	98 "computing directory digests. "
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	99 "Note that mode bits on symbolic links itself are not "
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	100 "considered.")
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	101 gp.add_argument(
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	102 "--generator", choices=("normal", "full", "none"),
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	103 default="normal",
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	104 help="""Put a `GENERATOR' line into the output.
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	105 `full' prints full Python and OS/platform version information,
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	106 `normal' prints just whether Python 2 or Python 3 is used, and `none'
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	107 suppresses the output completely. The default is `normal'.""")
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	108 gp.add_argument(
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	109 "--grouping-separator", action="store", dest="grouping_separator",
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	110 metavar="GROUPING-SEPARATOR",
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	111 help="""
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	112 Use the given GROUPING-SEPARATOR as thousands separator.
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	113 Use an empty GROUPING-SEPARATOR to disable grouping.
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	114 The effective default depends on the --output-style: for tagged output
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	115 it is the underscore `_', for tabbed output it is the dot `.'.
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	116
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	117 """)
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	118 gp.add_argument(
271 6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	119 "--half", "-H", action=SymlinkAction, dest="follow_symlinks",
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	120 const=FollowSymlinkConfig(True, False, False),
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	121 help="""Follow symbolic links given on the command line but do
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	122 not follow symlinks while traversing the directory tree.
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	123 Overwrites any other symlink related options
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	124 (--physical, --logical, -p, --no-follow-directory-symlinks,
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	125 --no-follow-file-symlinks, et al.).""")
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	126 gp.add_argument(
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	127 "--include", "-I", action=PatternMatchAction, kind="include",
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	128 dest="fnmatch_filters", metavar="PATTERN",
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	129 help="""Include names matching the given PATTERN.
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	130 For help on PATTERN use \"help patterns\".
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	131 Can be given more than once.""")
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	132 gp.add_argument(
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	133 "--logical", "-L", action=SymlinkAction, dest="follow_symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	134 const=FollowSymlinkConfig(True, True, True),
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	135 help="""Follow symbolic links everywhere: on command line
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	136 arguments and -- while walking -- directory and file symbolic links.
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	137 Overwrites any other symlink related options
271 6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	138 (--physical, --half, -p, --no-follow-directory-symlinks,
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	139 --no-follow-file-symlinks, et al.).""")
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	140 gp.add_argument(
150 f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	141 "--minimal", nargs="?", const="", default=None, metavar="TAG",
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	142 help="Produce minimal output only. If a TAG is given and not "
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	143 "empty use it as the leading \"ROOT (<TAG>)\" output.")
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	144 gp.add_argument(
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	145 "--mmap", action="store_true", dest="mmap", default=None,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	146 help="Use mmap if available. Default is to determine "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	147 "automatically from the filesize.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	148 gp.add_argument(
157 27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	149 "--mode", action="store_true", dest="metadata_mode",
158 d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	150 help="Consider the permission bits of stat(2) using S_IMODE (i.e. "
157 27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	151 "all bits without the filetype bits) when "
158 d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	152 "computing directory digests. Note that mode bits on "
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	153 "symbolic links itself are not considered.")
157 27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	154 gp.add_argument(
151 b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	155 "--mtime", action="store_true", dest="metadata_mtime",
b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	156 help="Consider the mtime of files (non-directories) when "
b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	157 "generating digests for directories. Digests for files are "
b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	158 "not affected.")
b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	159 gp.add_argument(
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	160 "--no-follow-directory-symlinks", action=SymlinkAction,
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	161 const="no-follow-directory-symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	162 dest="follow_symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	163 help="""Do not follow symbolic links to directories when walking a
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	164 directory tree. Augments --logical.""")
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	165 gp.add_argument(
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	166 "--no-follow-file-symlinks", action=SymlinkAction,
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	167 const="no-follow-file-symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	168 dest="follow_symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	169 help="""Dont follow symbolic links to files when walking a
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	170 directory tree. Augments --logical and -p.""")
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	171 gp.add_argument(
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	172 "--no-mmap", action="store_false", dest="mmap", default=None,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	173 help="Dont use mmap.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	174 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	175 "--output", "-o", action="store", metavar="OUTPUT",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	176 help="Put the checksum into given file. "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	177 "If not given or if it is given as `-' then stdout is used.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	178 gp.add_argument(
327 8e62738e7469 treesum: implement option "--output-style". Franz Glasner <fzglas.hg@dom66.de> parents: 326 diff changeset	179 "--output-style", dest="output_style", default="tagged",
330 61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	180 choices=("tagged", "tag", "tabular", "tab"),
327 8e62738e7469 treesum: implement option "--output-style". Franz Glasner <fzglas.hg@dom66.de> parents: 326 diff changeset	181 help="""
330 61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	182 Select the output style: "tagged" or "tag" selects a more BSD style tagged
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	183 format. "tabular" or "tab" select a more GNU style tabular format.
327 8e62738e7469 treesum: implement option "--output-style". Franz Glasner <fzglas.hg@dom66.de> parents: 326 diff changeset	184 Default is "tagged".
8e62738e7469 treesum: implement option "--output-style". Franz Glasner <fzglas.hg@dom66.de> parents: 326 diff changeset	185 """)
8e62738e7469 treesum: implement option "--output-style". Franz Glasner <fzglas.hg@dom66.de> parents: 326 diff changeset	186 gp.add_argument(
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	187 "--physical", "-P", action=SymlinkAction, dest="follow_symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	188 const=FollowSymlinkConfig(False, False, False),
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	189 help="""Do not follow any symbolic links whether they are given
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	190 on the command line or when walking the directory tree.
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	191 Overwrites any other symlink related options
271 6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	192 (--logical, --half, -p, --follow-directory-symlinks, --follow-file-symlinks,
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	193 et al.).""")
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	194 gp.add_argument(
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	195 "-p", action=SymlinkAction, dest="follow_symlinks",
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	196 const=FollowSymlinkConfig(False, False, True),
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	197 help="""Do not follow any symbolic links to directories,
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	198 whether they are given on the command line or when walking the directory tree,
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	199 but follow symbolic links to files.
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	200 Overwrites any other symlink related options
271 6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	201 (--logical, --half, --physical, --follow-directory-symlinks,
6fe88de236cb treesum: Implement the --half/-H option: follow symlinks given on the command line but no other symlinks. Franz Glasner <fzglas.hg@dom66.de> parents: 270 diff changeset	202 --no-follow-file-symlinks, et al.).
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	203 This is the default.""")
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	204 gp.add_argument(
168 bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	205 "--print-size", action="store_true",
bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	206 help="""Print the size of a file or the accumulated sizes of
bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	207 directory content into the output also.
bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	208 The size is not considered when computing digests. For symbolic links
bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	209 the size is not printed also.""")
bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	210 gp.add_argument(
170 8945be6b404e Mode for treesum.py to print only the size of files and the accumulated size of a directory: --size-only. Franz Glasner <fzglas.hg@dom66.de> parents: 169 diff changeset	211 "--size-only", action="store_true",
8945be6b404e Mode for treesum.py to print only the size of files and the accumulated size of a directory: --size-only. Franz Glasner <fzglas.hg@dom66.de> parents: 169 diff changeset	212 help="""Print only the size of files and for each directory its
8945be6b404e Mode for treesum.py to print only the size of files and the accumulated size of a directory: --size-only. Franz Glasner <fzglas.hg@dom66.de> parents: 169 diff changeset	213 accumulated directory size. Digests are not computed.""")
8945be6b404e Mode for treesum.py to print only the size of files and the accumulated size of a directory: --size-only. Franz Glasner <fzglas.hg@dom66.de> parents: 169 diff changeset	214 gp.add_argument(
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	215 "--size-width", action="store", type=int, metavar="SIZE-WIDTH",
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	216 dest="size_column_width", default=15,
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	217 help="""Some output styles print the a filesize right-aligned
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	218 in a column. SIZE-WIDTH is the (minimum) width to be used. The width includes
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	219 grouping separators. Use 0 if no alignment should be done. Default is 15.""")
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	220 gp.add_argument(
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	221 "--utf8", "--utf-8", action="store_true",
b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	222 help="""Encode all file paths using UTF-8 instead of
b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	223 the filesystem encoding. Add some error tag into the path if it cannot
b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	224 representated in Unicode cleanly.""")
b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	225 gp.add_argument(
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	226 "directories", nargs="*", metavar="DIRECTORY")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	227
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	228 def _populate_info_arguments(ip):
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	229 ip.add_argument(
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	230 "--last", action="store_true", dest="print_only_last_block",
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	231 help="Print only the last block of every given input file")
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	232 ip.add_argument(
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	233 "digest_files", nargs="+", metavar="TREESUM-DIGEST-FILE")
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	234
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	235 parser = argparse.ArgumentParser(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	236 description="Generate and verify checksums for directory trees.",
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	237 fromfile_prefix_chars='@',
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	238 add_help=False)
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	239
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	240 #
153 3505406ef9f3 Comment Franz Glasner <fzglas.hg@dom66.de> parents: 152 diff changeset	241 # Global options for all sub-commands.
3505406ef9f3 Comment Franz Glasner <fzglas.hg@dom66.de> parents: 152 diff changeset	242 # In a group because this allows a customized title.
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	243 #
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	244 gparser = parser.add_argument_group(title="Global Options")
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	245 gparser.add_argument(
191 1b8bc876146a Make "--debug" a global argument in treeview Franz Glasner <fzglas.hg@dom66.de> parents: 190 diff changeset	246 "--debug", action="store_true",
1b8bc876146a Make "--debug" a global argument in treeview Franz Glasner <fzglas.hg@dom66.de> parents: 190 diff changeset	247 help="Activate debug logging to stderr")
1b8bc876146a Make "--debug" a global argument in treeview Franz Glasner <fzglas.hg@dom66.de> parents: 190 diff changeset	248 gparser.add_argument(
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	249 "-v", "--version", action="version",
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	250 version="%s (rv:%s)" % (__version__, __revision__),
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	251 help="Show program's version number and exit")
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	252 gparser.add_argument(
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	253 "-h", "--help", action="help",
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	254 help="Show this help message and exit")
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	255
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	256 #
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	257 # Subcommands
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	258 #
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	259 subparsers = parser.add_subparsers(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	260 dest="subcommand",
145 073e0faea599 Optimize help output for subcommands Franz Glasner <fzglas.hg@dom66.de> parents: 144 diff changeset	261 title="Commands",
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	262 description="This tool uses subcommands. "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	263 "To see detailed help for a specific subcommand use "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	264 "the -h/--help option after the subcommand name. "
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	265 "A list of valid commands and their short descriptions "
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	266 "is listed below:",
145 073e0faea599 Optimize help output for subcommands Franz Glasner <fzglas.hg@dom66.de> parents: 144 diff changeset	267 metavar="COMMAND")
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	268
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	269 genparser = subparsers.add_parser(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	270 "generate",
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	271 help="Generate checksums for directory trees",
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	272 description="Generate checksums for directory trees.")
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	273 _populate_generate_arguments(genparser)
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	274 # And an alias for "generate"
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	275 genparser2 = subparsers.add_parser(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	276 "gen",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	277 help="Alias for \"generate\"",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	278 description="Generate checksums for directory trees. "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	279 "This is an alias to \"generate\".")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	280 _populate_generate_arguments(genparser2)
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	281
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	282 infoparser = subparsers.add_parser(
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	283 "info",
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	284 help="Print some information from given treesum digest file",
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	285 description="""Print some informations from given treesum digest files
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	286 to stdout."""
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	287 )
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	288 _populate_info_arguments(infoparser)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	289
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	290 hparser = subparsers.add_parser(
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	291 "help",
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	292 help="Show this help message or a subcommand's help and exit",
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	293 description="Show this help message or a subcommand's help and exit.")
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	294 hparser.add_argument("help_command", nargs='?', metavar="COMMAND")
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	295
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	296 patparser = subparsers.add_parser(
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	297 "patterns",
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	298 help="Show the help for PATTERNs and exit",
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	299 description=fnmatch.HELP_DESCRIPTION,
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	300 formatter_class=argparse.RawDescriptionHelpFormatter)
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	301
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	302 vparser = subparsers.add_parser(
144 b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	303 "version",
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	304 help="Show the program's version number and exit",
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	305 description="Show the program's version number and exit.")
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	306
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	307 # Parse leniently to just check for "version" and/or help
146 7d8df8311e3b Optimize argument parsing for the "version" command Franz Glasner <fzglas.hg@dom66.de> parents: 145 diff changeset	308 opts, _dummy = parser.parse_known_args(args=argv)
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	309
144 b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	310 if opts.subcommand == "version":
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	311 print("%s (rv:%s)" % (__version__, __revision__),
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	312 file=sys.stdout)
174 fc1055878775 Use "return 0" instead of "sys.exit(0)" when printing version and help# Franz Glasner <fzglas.hg@dom66.de> parents: 173 diff changeset	313 return 0
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	314 if opts.subcommand == "help":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	315 if not opts.help_command:
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	316 parser.print_help()
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	317 else:
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	318 if opts.help_command == "generate":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	319 genparser.print_help()
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	320 elif opts.help_command == "gen":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	321 genparser2.print_help()
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	322 elif opts.help_command == "info":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	323 infoparser.print_help()
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	324 elif opts.help_command == "version":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	325 vparser.print_help()
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	326 elif opts.help_command == "help":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	327 hparser.print_help()
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	328 elif opts.help_command == "patterns":
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	329 patparser.print_help()
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	330 else:
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	331 parser.print_help()
174 fc1055878775 Use "return 0" instead of "sys.exit(0)" when printing version and help# Franz Glasner <fzglas.hg@dom66.de> parents: 173 diff changeset	332 return 0
144 b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	333
146 7d8df8311e3b Optimize argument parsing for the "version" command Franz Glasner <fzglas.hg@dom66.de> parents: 145 diff changeset	334 # Reparse strictly
7d8df8311e3b Optimize argument parsing for the "version" command Franz Glasner <fzglas.hg@dom66.de> parents: 145 diff changeset	335 opts = parser.parse_args(args=argv)
7d8df8311e3b Optimize argument parsing for the "version" command Franz Glasner <fzglas.hg@dom66.de> parents: 145 diff changeset	336
176 7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	337 # Minimal logging -- just for debugging - not for more "normal" use
7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	338 logging.basicConfig(
7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	339 level=logging.DEBUG if opts.debug else logging.WARNING,
7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	340 stream=sys.stderr,
7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	341 format="[%(asctime)s][%(levelname)s][%(process)d:%(name)s] %(message)s"
7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	342 )
7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	343 logging.captureWarnings(True)
7f5d05a625fd Implement preconditions for some debug logging Franz Glasner <fzglas.hg@dom66.de> parents: 174 diff changeset	344
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	345 return treesum(opts, patparser=patparser)
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	346
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	347
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	348 FollowSymlinkConfig = collections.namedtuple("FollowSymlinkConfig",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	349 ["command_line",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	350 "directory",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	351 "file"])
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	352
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	353
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	354 class SymlinkAction(argparse.Action):
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	355
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	356 """`type' is fixed here.
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	357 `dest' is a tuple with three items:
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	358
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	359 1. follow symlinks on the command line
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	360 2. follow directory symlinks while walking
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	361 3. follow file symlinks while walking (not yet implemented)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	362
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	363 """
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	364
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	365 def __init__(self, args, *kwargs):
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	366 if "nargs" in kwargs:
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	367 raise ValueError("`nargs' not allowed")
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	368 if "type" in kwargs:
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	369 raise ValueError("`type' not allowed")
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	370 c = kwargs.get("const", None)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	371 if c is None:
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	372 raise ValueError("a const value is needed")
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	373 if (not isinstance(c, FollowSymlinkConfig)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	374 and c not in ("follow-directory-symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	375 "no-follow-directory-symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	376 "follow-file-symlinks",
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	377 "no-follow-file-symlinks")):
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	378 raise ValueError(
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	379 "invalid value for the `const' configuration value")
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	380 default = kwargs.get("default", None)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	381 if (default is not None
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	382 and not isinstance(default, FollowSymlinkConfig)):
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	383 raise TypeError("invalid type for `default'")
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	384 kwargs["nargs"] = 0
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	385 super(SymlinkAction, self).__init__(args, *kwargs)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	386
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	387 def __call__(self, parser, namespace, values, option_string=None):
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	388 curval = getattr(namespace, self.dest, None)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	389 if curval is None:
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	390 curval = FollowSymlinkConfig(False, False, True)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	391 if isinstance(self.const, FollowSymlinkConfig):
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	392 curval = self.const
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	393 else:
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	394 if self.const == "follow-directory-symlinks":
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	395 curval = FollowSymlinkConfig(
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	396 curval.command_line, True, curval.file)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	397 elif self.const == "no-follow-directory-symlinks":
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	398 curval = FollowSymlinkConfig(
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	399 curval.command_line, False, curval.file)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	400 elif self.const == "follow-file-symlinks":
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	401 curval = FollowSymlinkConfig(
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	402 curval.command_line, curval.directory, True)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	403 elif self.const == "no-follow-file-symlinks":
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	404 curval = FollowSymlinkConfig(
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	405 curval.command_line, curval.directory, False)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	406 else:
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	407 assert False, "Implementation error: not yet implemented"
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	408
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	409 setattr(namespace, self.dest, curval)
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	410
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	411
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	412 class PatternMatchAction(argparse.Action):
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	413
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	414 def __init__(self, args, *kwargs):
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	415 if "nargs" in kwargs:
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	416 raise argparse.ArgumentError(None, "`nargs' not allowed")
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	417 if "type" in kwargs:
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	418 raise argparse.ArgumentError(None, "`type' not allowed")
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	419 kwargs["nargs"] = 1
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	420
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	421 self.__kind = kwargs.pop("kind", None)
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	422 if self.__kind is None:
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	423 raise argparse.ArgumentError(None, "`kind' is required")
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	424 if self.__kind not in ("exclude", "include", "accept-treesum"):
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	425 raise argparse.ArgumentError(
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	426 None,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	427 "`kind' must be one of `include', `exclude' or"
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	428 " `accept-treesum'")
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	429
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	430 super(PatternMatchAction, self).__init__(args, *kwargs)
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	431
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	432 def __call__(self, parser, namespace, values, option_string=None):
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	433 items = getattr(namespace, self.dest, None)
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	434 if items is None:
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	435 items = []
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	436 setattr(namespace, self.dest, items)
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	437 for v in values:
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	438 items.append((self.__kind, v))
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	439
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	440
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	441 def gen_generate_opts(directories=[],
172 804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	442 algorithm=util.default_algotag(),
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	443 append_output=False,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	444 base64=False,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	445 comment=[],
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	446 fnmatch_filters=[],
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	447 follow_symlinks=FollowSymlinkConfig(False, False, False),
158 d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	448 full_mode=False,
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	449 generator="normal",
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	450 grouping_separator=None, # the output writer selects
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	451 logical=None,
150 f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	452 minimal=None,
157 27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	453 mode=False,
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	454 mmap=None,
151 b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	455 mtime=False,
168 bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	456 output=None,
327 8e62738e7469 treesum: implement option "--output-style". Franz Glasner <fzglas.hg@dom66.de> parents: 326 diff changeset	457 output_style="tagged",
170 8945be6b404e Mode for treesum.py to print only the size of files and the accumulated size of a directory: --size-only. Franz Glasner <fzglas.hg@dom66.de> parents: 169 diff changeset	458 print_size=False,
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	459 size_only=False,
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	460 size_column_width=15,
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	461 utf8=False):
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	462 if not isinstance(follow_symlinks, FollowSymlinkConfig):
1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	463 raise TypeError("`follow_symlinks' must be a FollowSymlinkConfig")
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	464 if not isinstance(fnmatch_filters, (list, tuple, type(None))):
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	465 raise TypeError("`fnmatch_filters' must be a sequence (list, tuple)")
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	466 if fnmatch_filters:
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	467 for f in fnmatch_filters:
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	468 if not isinstance(f, (tuple, list)):
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	469 raise TypeError(
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	470 "items in `fnmatch_filters' must be tuples or lists")
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	471 if f[0] not in ("exclude", "include"):
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	472 raise ValueError(
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	473 "every kind of every item in `fnmatch_filters' must be"
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	474 " \"include\", \"exclude\" or \"accept-treesum\""
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	475 )
328 97bba2295eb3 treesum: parameter validation for "generator" Franz Glasner <fzglas.hg@dom66.de> parents: 327 diff changeset	476 if generator not in ("normal", "full", "none"):
97bba2295eb3 treesum: parameter validation for "generator" Franz Glasner <fzglas.hg@dom66.de> parents: 327 diff changeset	477 raise ValueError("given generator `%s' not allowed" % (generator, ))
330 61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	478 if output_style not in ("tagged", "tag", "tabular", "tab"):
327 8e62738e7469 treesum: implement option "--output-style". Franz Glasner <fzglas.hg@dom66.de> parents: 326 diff changeset	479 raise ValueError(
8e62738e7469 treesum: implement option "--output-style". Franz Glasner <fzglas.hg@dom66.de> parents: 326 diff changeset	480 "given output_style `%s' not allowed" % (output_style,))
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	481
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	482 # Not following symlinks to files is not yet supported: reset to True
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	483 # if not follow_symlinks.file:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	484 # follow_symlinks = follow_symlinks._make([follow_symlinks.command_line,
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	485 # follow_symlinks.directory,
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	486 # True])
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	487 # logging.warning("Coercing to follow-symlinks-file")
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	488 opts = argparse.Namespace(
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	489 directories=directories,
172 804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	490 algorithm=util.argv2algo(algorithm),
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	491 append_output=append_output,
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	492 base64=base64,
135 dbf27681a1f6 Allow to put comments into the output with "--comment" Franz Glasner <fzglas.hg@dom66.de> parents: 134 diff changeset	493 comment=comment,
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	494 fnmatch_filters=fnmatch_filters,
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	495 follow_symlinks=follow_symlinks,
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	496 generator=generator,
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	497 grouping_separator=grouping_separator,
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	498 logical=logical,
150 f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	499 minimal=minimal,
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	500 mmap=mmap,
158 d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	501 metadata_full_mode=full_mode,
157 27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	502 metadata_mode=mode,
151 b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	503 metadata_mtime=mtime,
168 bcc4441cf216 Implement "--print-size" to print file and accumulated directory sizes also. Franz Glasner <fzglas.hg@dom66.de> parents: 161 diff changeset	504 output=output,
327 8e62738e7469 treesum: implement option "--output-style". Franz Glasner <fzglas.hg@dom66.de> parents: 326 diff changeset	505 output_style=output_style,
170 8945be6b404e Mode for treesum.py to print only the size of files and the accumulated size of a directory: --size-only. Franz Glasner <fzglas.hg@dom66.de> parents: 169 diff changeset	506 print_size=print_size,
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	507 size_only=size_only,
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	508 size_column_width=size_column_width,
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	509 utf8=utf8)
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	510 return opts
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	511
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	512
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	513 def gen_info_opts(digest_files=[], last=False):
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	514 opts = argparse.Namespace(
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	515 digest_files=digest_files,
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	516 print_only_last_block=last)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	517 return opts
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	518
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	519
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	520 def treesum(opts, patparser=None):
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	521 # XXX TBD: opts.check and opts.checklist (as in shasum.py)
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	522 if opts.subcommand in ("generate", "gen"):
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	523 return generate_treesum(opts)
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	524 elif opts.subcommand == "info":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	525 return print_treesum_digestfile_infos(opts)
300 1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	526 elif opts.subcommand == "patterns":
1fc117f5f9a1 treesum: Implement --include/--exclude commandline parsing for file name inclusion and exclusion. Franz Glasner <fzglas.hg@dom66.de> parents: 299 diff changeset	527 patparser.print_help()
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	528 else:
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	529 raise RuntimeError(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	530 "command `{}' not yet handled".format(opts.subcommand))
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	531
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	532
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	533 def generate_treesum(opts):
149 f717854be1de Put the defaults generation when generating directory digests into "generate_treesum()" instead of "main()" Franz Glasner <fzglas.hg@dom66.de> parents: 148 diff changeset	534 # Provide defaults
f717854be1de Put the defaults generation when generating directory digests into "generate_treesum()" instead of "main()" Franz Glasner <fzglas.hg@dom66.de> parents: 148 diff changeset	535 if not opts.algorithm:
172 804a823c63f5 Now the selection of the default algorithm depends on availiability in hashlib Franz Glasner <fzglas.hg@dom66.de> parents: 170 diff changeset	536 opts.algorithm = util.argv2algo(util.default_algotag())
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	537 if not opts.directories:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	538 opts.directories.append(".")
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	539
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	540 if opts.output is None or opts.output == "-":
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	541 if hasattr(sys.stdout, "buffer"):
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	542 out_cm = cm.nullcontext(sys.stdout.buffer)
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	543 else:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	544 out_cm = cm.nullcontext(sys.stdout)
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	545 else:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	546 if opts.append_output:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	547 out_cm = open(opts.output, "ab")
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	548 else:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	549 out_cm = open(opts.output, "wb")
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	550
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	551 fnmatcher = fnmatch.FnMatcher.build_from_commandline_patterns(
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	552 opts.fnmatch_filters)
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	553
330 61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	554 if opts.output_style in ("tagged", "tag"):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	555 writerstyle = TaggedTreesumWriter
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	556 elif opts.output_style in ("tabular", "tab"):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	557 writerstyle = TabularTreesumWriter
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	558 else:
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	559 raise NotImplementedError("`output_style'")
327 8e62738e7469 treesum: implement option "--output-style". Franz Glasner <fzglas.hg@dom66.de> parents: 326 diff changeset	560
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	561 with out_cm as outfp:
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	562 writer = writerstyle(outfp,
336 0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	563 size_only=opts.size_only,
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	564 print_size=opts.print_size,
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	565 use_base64=opts.base64,
341 728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	566 grouping_separator=opts.grouping_separator,
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	567 size_column_width=opts.size_column_width,
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	568 )
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	569 for d in opts.directories:
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	570 V1DirectoryTreesumGenerator(
331 9ee84624587f treesum: move the handling of base64 digest output into the writers completely Franz Glasner <fzglas.hg@dom66.de> parents: 330 diff changeset	571 opts.algorithm, opts.mmap,
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	572 opts.follow_symlinks,
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	573 opts.generator,
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	574 opts.metadata_mode,
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	575 opts.metadata_full_mode,
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	576 opts.metadata_mtime,
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	577 opts.size_only,
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	578 opts.print_size,
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	579 opts.utf8,
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	580 minimal=opts.minimal,
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	581 fnmatcher=fnmatcher).generate(
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	582 writer, d, comment=opts.comment)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	583
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	584
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	585 class V1DirectoryTreesumGenerator(object):
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	586
331 9ee84624587f treesum: move the handling of base64 digest output into the writers completely Franz Glasner <fzglas.hg@dom66.de> parents: 330 diff changeset	587 def __init__(self, algorithm, use_mmap,
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	588 follow_symlinks,
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	589 with_generator,
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	590 with_metadata_mode, with_metadata_full_mode,
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	591 with_metadata_mtime, size_only, print_size, utf8_mode,
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	592 minimal=None,
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	593 fnmatcher=None):
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	594 super(V1DirectoryTreesumGenerator, self).__init__()
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	595 self._algorithm = algorithm
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	596 self._use_mmap = use_mmap
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	597 self._follow_symlinks = follow_symlinks
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	598 self._with_generator = with_generator
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	599 self._with_metadata_mode = with_metadata_mode
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	600 self._with_metadata_full_mode = with_metadata_full_mode
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	601 self._with_metadata_mtime = with_metadata_mtime
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	602 self._size_only = size_only
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	603 self._print_size = print_size
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	604 self._utf8_mode = utf8_mode
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	605 self._minimal = minimal
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	606 self._fnmatcher = fnmatcher
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	607
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	608 def generate(self, writer, root, comment=None):
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	609 """
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	610
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	611 :param outfp: a binary file with a "write()" and a "flush()" method
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	612
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	613 """
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	614 self._writer = writer
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	615 self._writer.start("1")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	616 self._writer.write_fsencoding(util.n(walk.getfsencoding().upper()))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	617 self._writer.flush()
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	618
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	619 if self._with_generator == "none":
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	620 pass # do nothing
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	621 elif self._with_generator == "normal":
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	622 self._writer.write_generator("PY2" if util.PY2 else "PY3")
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	623 elif self._with_generator == "full":
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	624 import platform
295 4a259fb9968e treesum: Add the treesum version and revision to the full generator output Franz Glasner <fzglas.hg@dom66.de> parents: 291 diff changeset	625 info = ("treesum %s (rv:%s), %s %s, %s"
4a259fb9968e treesum: Add the treesum version and revision to the full generator output Franz Glasner <fzglas.hg@dom66.de> parents: 291 diff changeset	626 % (__version__,
4a259fb9968e treesum: Add the treesum version and revision to the full generator output Franz Glasner <fzglas.hg@dom66.de> parents: 291 diff changeset	627 __revision__,
4a259fb9968e treesum: Add the treesum version and revision to the full generator output Franz Glasner <fzglas.hg@dom66.de> parents: 291 diff changeset	628 platform.python_implementation(),
4a259fb9968e treesum: Add the treesum version and revision to the full generator output Franz Glasner <fzglas.hg@dom66.de> parents: 291 diff changeset	629 platform.python_version(),
4a259fb9968e treesum: Add the treesum version and revision to the full generator output Franz Glasner <fzglas.hg@dom66.de> parents: 291 diff changeset	630 platform.platform()))
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	631 self._writer.write_generator(info)
205 63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	632 else:
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	633 raise NotImplementedError(
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	634 "not implemented: %s" % (self._with_generator,))
63088d3675bb Add a "GENERATOR" line with control flats to treesum.py Franz Glasner <fzglas.hg@dom66.de> parents: 204 diff changeset	635
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	636 #
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	637 # Note: Given non-default flags that are relevant for
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	638 # directory traversal.
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	639 #
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	640 flags = []
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	641 if self._with_metadata_full_mode:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	642 flags.append("with-metadata-fullmode")
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	643 elif self._with_metadata_mode:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	644 flags.append("with-metadata-mode")
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	645 if self._with_metadata_mtime:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	646 flags.append("with-metadata-mtime")
219 19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	647 flags.append("follow-symlinks-commandline"
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	648 if self._follow_symlinks.command_line
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	649 else "no-follow-symlinks-commandline")
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	650 flags.append("follow-symlinks-directory"
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	651 if self._follow_symlinks.directory
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	652 else "no-follow-symlinks-directory")
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	653 flags.append("follow-symlinks-file"
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	654 if self._follow_symlinks.file
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	655 else "no-follow-symlinks-file")
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	656 if self._size_only:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	657 flags.append("size-only")
219 19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	658 flags.append("utf8-encoding" if self._utf8_mode else "fs-encoding")
211 5bb0b25f8e99 FIX: FLAGS output: wrong "else" for "utf-8-mode" and "print-size" Franz Glasner <fzglas.hg@dom66.de> parents: 210 diff changeset	659 if self._print_size:
5bb0b25f8e99 FIX: FLAGS output: wrong "else" for "utf-8-mode" and "print-size" Franz Glasner <fzglas.hg@dom66.de> parents: 210 diff changeset	660 flags.append("print-size")
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	661 self._writer.write_flags(flags)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	662
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	663 if self._minimal is None:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	664 # Write execution timestamps in POSIX epoch and ISO format
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	665 ts = int(time.time())
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	666 self._writer.write_timestamp(ts)
291 cc2deeb5f2e6 treesum: Indicate that the ISO timestamp is UTC Franz Glasner <fzglas.hg@dom66.de> parents: 290 diff changeset	667 ts = (datetime.datetime.utcfromtimestamp(ts)).isoformat("T") + "Z"
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	668 self._writer.write_isotimestamp(ts)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	669
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	670 if comment:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	671 for line in comment:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	672 self._writer.write_comment(line)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	673
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	674 for action, kind, pattern in self._fnmatcher.definitions():
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	675 self._writer.write_fnmatch_pattern(action, kind, pattern)
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	676
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	677 if self._minimal is not None:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	678 self._writer.write_root(
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	679 (walk.WalkDirEntry.alt_u8(self._minimal)
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	680 if self._minimal else b""))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	681 else:
278 822cf3a1da22 treesum: FIX: Removal of backslashes in output: do this only where really needed. Franz Glasner <fzglas.hg@dom66.de> parents: 277 diff changeset	682 self._writer.write_root(walk.WalkDirEntry.alt_u8(
822cf3a1da22 treesum: FIX: Removal of backslashes in output: do this only where really needed. Franz Glasner <fzglas.hg@dom66.de> parents: 277 diff changeset	683 util.normalize_filename(root, True)))
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	684
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	685 self._writer.flush()
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	686
210 1be3af138183 Refactor option handling for configuring symlink handling: now all variations are supported Franz Glasner <fzglas.hg@dom66.de> parents: 208 diff changeset	687 if not self._follow_symlinks.command_line and os.path.islink(root):
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	688 linktgt = walk.WalkDirEntry.from_readlink(os.readlink(root))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	689 linkdgst = self._algorithm[0]()
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	690 linkdgst.update(linktgt.fspath)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	691 dir_dgst = self._algorithm[0]()
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	692 dir_dgst.update(b"2:L@,")
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	693 dir_dgst.update(
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	694 util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	695 b"%d:%s,%d:%s,",
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	696 len(self._algorithm[1]), util.b(self._algorithm[1]),
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	697 len(linkdgst.digest()), linkdgst.digest()))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	698 if self._size_only:
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	699 self._writer.write_size(b"./@/", -1)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	700 else:
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	701 sz = -1 if self._print_size else None
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	702 self._writer.write_file_digest(
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	703 self._algorithm[1],
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	704 b"./@/",
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	705 dir_dgst.digest(),
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	706 sz)
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	707 self._writer.flush()
268 8aadffaaad5f treesum: refactor: remove early return and use an "else" branch instead Franz Glasner <fzglas.hg@dom66.de> parents: 267 diff changeset	708 else:
8aadffaaad5f treesum: refactor: remove early return and use an "else" branch instead Franz Glasner <fzglas.hg@dom66.de> parents: 267 diff changeset	709 self._generate(os.path.normpath(root), tuple())
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	710 self._writer.finish()
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	711
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	712 def _generate(self, root, top):
301 d246b631b85a treesum: Change debug output: make output when entering a new root directory different from recursing into a sub-directory Franz Glasner <fzglas.hg@dom66.de> parents: 300 diff changeset	713 if top:
d246b631b85a treesum: Change debug output: make output when entering a new root directory different from recursing into a sub-directory Franz Glasner <fzglas.hg@dom66.de> parents: 300 diff changeset	714 logging.debug("Recursing into directory: %s/%r", root, top)
d246b631b85a treesum: Change debug output: make output when entering a new root directory different from recursing into a sub-directory Franz Glasner <fzglas.hg@dom66.de> parents: 300 diff changeset	715 else:
d246b631b85a treesum: Change debug output: make output when entering a new root directory different from recursing into a sub-directory Franz Glasner <fzglas.hg@dom66.de> parents: 300 diff changeset	716 logging.debug("Handling root directory: %s", root)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	717 path = os.path.join(root, *top) if top else root
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	718 # Determine also the path to be used for directory filtering
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	719 fpath = join_output_path(top, None) if top else ""
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	720 if self._fnmatcher:
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	721 if not self._fnmatcher.shall_visit(fpath):
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	722 logging.debug("Skipping directory: %s", fpath)
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	723 return (None, None, None, None)
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	724 try:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	725 with walk.ScanDir(path) as dirscan:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	726 fsobjects = list(dirscan)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	727 except OSError as e:
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	728 #
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	729 # NOTE: Sync the error handler code with this method's
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	730 # code below before returning!
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	731 #
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	732 if e.errno == errno.ENOTDIR:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	733 # object exists but is not a directory
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	734 errmsg = b"not a directory"
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	735 elif e.errno in (errno.EACCES, errno.EPERM,
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	736 getattr(errno, "ENOTCAPABLE", errno.EACCES)):
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	737 # no permissions
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	738 errmsg = (
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	739 b"access denied / no permissions / missing capabilities")
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	740 elif e.errno == errno.ENOENT:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	741 # given object does not exist
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	742 errmsg = b"no such file or directory"
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	743 else:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	744 raise
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	745 if self._utf8_mode:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	746 opath = walk.WalkDirEntry.alt_u8(path)
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	747 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	748 opath = walk.WalkDirEntry.alt_fs(path)
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	749 self._writer.write_error(util.interpolate_bytes(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	750 b"`%s': %s", opath, errmsg))
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	751 # Reuse from top
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	752 opath = join_output_path(top, None)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	753 if opath:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	754 if self._utf8_mode:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	755 opath = walk.WalkDirEntry.alt_u8(opath)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	756 else:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	757 opath = walk.WalkDirEntry.alt_fs(opath)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	758 if self._size_only:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	759 self._writer.write_size(opath, None)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	760 else:
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	761 self._writer.write_file_digest(
e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	762 self._algorithm[1], opath, None, None)
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	763 self._writer.flush()
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	764 return (e.errno, None, None, None)
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	765
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	766 # Check whether to accept existing treesum digest files
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	767 if self._fnmatcher:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	768 for fso in fsobjects:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	769 fpath = join_output_path(top, fso.name)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	770 if self._fnmatcher.shall_accept_treesum(fpath):
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	771 # Yes we have hit a treesum digest file
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	772 logging.debug("Accepting existing treesum from: %s", fpath)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	773 collector = DigestSizeCollector()
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	774 try:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	775 collector.collect_from_file(os.path.join(root, fpath))
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	776 except OSError as e:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	777 eno = e.errno
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	778 emsg = e.strerror
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	779 except Exception as e:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	780 # XXX FIXME: other EIO, EBADF, EFAULT
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	781 eno = errno.ESRCH
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	782 emsg = str(e)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	783 else:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	784 eno = 0
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	785 emsg = None
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	786 if self._utf8_mode:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	787 fpath = walk.WalkDirEntry.alt_u8(fpath)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	788 else:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	789 fpath = walk.WalkDirEntry.alt_fs(fpath)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	790 opath = join_output_path(top, None)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	791 if self._utf8_mode:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	792 opath = walk.WalkDirEntry.alt_u8(opath)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	793 else:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	794 opath = walk.WalkDirEntry.alt_fs(opath)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	795 if eno == 0:
313 dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	796 #
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	797 # treesum file could be read.
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	798 # Now check whether the infos we got from it are
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	799 # compatible with our current requirements
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	800 # (digest, size).
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	801 #
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	802 if self._size_only:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	803 if collector.size is None:
313 dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	804 #
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	805 # This is a severe error here: just the size
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	806 # is required, but we have not got one.
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	807 #
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	808 self._writer.write_error(util.b(
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	809 util.interpolate_bytes(
315 aaf0b3b53e48 treesum: FIX: format string for interpolate_bytes() must be of bytes type Franz Glasner <fzglas.hg@dom66.de> parents: 314 diff changeset	810 b"No size in treesum-file `%s' while"
aaf0b3b53e48 treesum: FIX: format string for interpolate_bytes() must be of bytes type Franz Glasner <fzglas.hg@dom66.de> parents: 314 diff changeset	811 b" requiring it",
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	812 fpath),
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	813 "utf-8"))
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	814 self._writer.write_size(opath, None)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	815 return (errno.ESRCH, None, None, None)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	816 else:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	817 if self._print_size:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	818 if collector.size is None:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	819 #
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	820 # XXX FIXME: Is this a severe error
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	821 # here? Currently: no
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	822 #
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	823 self._writer.write_error(util.b(
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	824 util.interpolate_bytes(
315 aaf0b3b53e48 treesum: FIX: format string for interpolate_bytes() must be of bytes type Franz Glasner <fzglas.hg@dom66.de> parents: 314 diff changeset	825 b"No size in treesum-file `%s'",
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	826 fpath),
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	827 "utf-8"))
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	828 sz = -1
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	829 else:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	830 sz = collector.size
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	831 else:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	832 sz = None
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	833 if collector.digest is None:
313 dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	834 #
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	835 # This is really a severe error. Most probably
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	836 # the treesum file was created with
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	837 # "--size-only" and contains no digest.
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	838 #
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	839 self._writer.write_error(util.b(
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	840 util.interpolate_bytes(
315 aaf0b3b53e48 treesum: FIX: format string for interpolate_bytes() must be of bytes type Franz Glasner <fzglas.hg@dom66.de> parents: 314 diff changeset	841 b"No digest in treesum-file `%s' while"
aaf0b3b53e48 treesum: FIX: format string for interpolate_bytes() must be of bytes type Franz Glasner <fzglas.hg@dom66.de> parents: 314 diff changeset	842 b" it is required",
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	843 fpath),
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	844 "utf-8"))
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	845 self._writer.write_file_digest(
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	846 collector.algorithm or "MD5",
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	847 opath,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	848 None,
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	849 sz)
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	850 return (errno.ESRCH, None, None, None)
313 dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	851 # We got all required infos without errors
317 fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	852 self._writer.write_accept_treesum_file(fpath)
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	853 if self._size_only:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	854 self._writer.write_size(opath, collector.size)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	855 else:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	856 self._writer.write_file_digest(
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	857 collector.algorithm, opath, collector.digest,
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	858 sz)
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	859 return (0,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	860 collector.algorithm,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	861 collector.digest,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	862 collector.size)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	863 else:
313 dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	864 #
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	865 # treesum file could not be read
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	866 #
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	867 self._writer.write_error(util.interpolate_bytes(
315 aaf0b3b53e48 treesum: FIX: format string for interpolate_bytes() must be of bytes type Franz Glasner <fzglas.hg@dom66.de> parents: 314 diff changeset	868 b"Cannot read treesum-file `%s' for directory"
aaf0b3b53e48 treesum: FIX: format string for interpolate_bytes() must be of bytes type Franz Glasner <fzglas.hg@dom66.de> parents: 314 diff changeset	869 b"`%s': %s",
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	870 fpath,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	871 opath,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	872 util.b(emsg, "utf-8")))
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	873 if self._size_only:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	874 self._writer.write_size(opath, None)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	875 else:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	876 self._writer.write_file_digest(
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	877 self._algorithm[1], opath, None, None)
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	878 return (eno, None, None, None)
313 dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	879 #
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	880 # No treesum file: just process normally with digesting
dbad01eb9d03 treesum: comments Franz Glasner <fzglas.hg@dom66.de> parents: 308 diff changeset	881 #
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	882 if self._utf8_mode:
221 ca9d5a0dc9bb Rename WalkDirEntry.sort_key and WalkDirEntry.alt_sort_key Franz Glasner <fzglas.hg@dom66.de> parents: 220 diff changeset	883 fsobjects.sort(key=walk.WalkDirEntry.sort_key_u8)
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	884 else:
221 ca9d5a0dc9bb Rename WalkDirEntry.sort_key and WalkDirEntry.alt_sort_key Franz Glasner <fzglas.hg@dom66.de> parents: 220 diff changeset	885 fsobjects.sort(key=walk.WalkDirEntry.sort_key_fs)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	886 dir_dgst = self._algorithm[0]()
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	887 dir_size = 0
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	888 dir_tainted = False
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	889 for fso in fsobjects:
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	890 # Determine the effective name to be used for digesting
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	891 if self._utf8_mode:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	892 if fso.u8name is None:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	893 dir_tainted = True
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	894 effective_fso_name = fso.alt_u8name
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	895 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	896 effective_fso_name = fso.u8name
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	897 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	898 if fso.fsname is None:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	899 dir_tainted = True
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	900 effective_fso_name = fso.alt_fsname
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	901 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	902 effective_fso_name = fso.fsname
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	903 # Determine the path (mostly its prefix) that is to be printed
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	904 opath = join_output_path(top, fso.name)
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	905 # Determine the path to be used for filename filtering
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	906 fpath = opath
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	907 if self._fnmatcher:
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	908 if not self._fnmatcher.shall_visit(fpath):
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	909 logging.debug("Skipping: %s", fpath)
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	910 continue
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	911 if self._utf8_mode:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	912 opath = walk.WalkDirEntry.alt_u8(opath)
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	913 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	914 opath = walk.WalkDirEntry.alt_fs(opath)
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	915 if fso.is_special:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	916 # Determine the tag character
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	917 if fso.is_chr:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	918 special_tag = b':'
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	919 elif fso.is_blk:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	920 special_tag = b';'
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	921 elif fso.is_fifo:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	922 special_tag = b'\|'
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	923 elif fso.is_socket:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	924 special_tag = b'='
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	925 elif fso.is_door:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	926 special_tag = b'>'
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	927 elif fso.is_whiteout:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	928 special_tag = b'%'
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	929 elif fso.is_eventport:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	930 special_tag = b'+'
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	931 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	932 assert False, "unknown special filesystem object"
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	933 assert fso.stat is not None # because .is_special is True
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	934 if fso.is_symlink and not self._follow_symlinks.file:
202 b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	935 linktgt = walk.WalkDirEntry.from_readlink(
b9b38584919b First preparations to implement an UTF-8-mode for treeview Franz Glasner <fzglas.hg@dom66.de> parents: 200 diff changeset	936 os.readlink(fso.path))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	937 linkdgst = self._algorithm[0]()
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	938 if self._utf8_mode:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	939 if linktgt.u8path is None:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	940 dir_tainted = True
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	941 linkdgst.update(linktgt.alt_u8path)
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	942 else:
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	943 linkdgst.update(linktgt.u8path)
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	944 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	945 if linktgt.fspath is None:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	946 dir_tainted = True
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	947 linkdgst.update(linktgt.alt_fspath)
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	948 else:
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	949 linkdgst.update(linktgt.fspath)
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	950 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	951 b"2:@%s,%d:%s,",
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	952 special_tag,
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	953 len(effective_fso_name),
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	954 effective_fso_name))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	955 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	956 b"%d:%s,%d:%s,",
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	957 len(self._algorithm[1]), util.b(self._algorithm[1]),
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	958 len(linkdgst.digest()), linkdgst.digest()))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	959 #
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	960 # - no mtime and no mode for symlinks
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	961 # - also does not count for dir_size
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	962 #
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	963 if self._size_only:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	964 self._writer.write_size(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	965 util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	966 b"%s/./@%s", opath, special_tag),
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	967 -1)
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	968 else:
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	969 sz = -1 if self._print_size else None
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	970 self._writer.write_file_digest(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	971 self._algorithm[1],
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	972 util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	973 b"%s/./@%s", opath, special_tag),
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	974 linkdgst.digest(),
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	975 sz)
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	976 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	977 #
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	978 # Follow the symlink to special file and/or handle a
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	979 # special file
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	980 #
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	981 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	982 b"1:%s,%d:%s,",
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	983 special_tag,
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	984 len(effective_fso_name),
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	985 effective_fso_name))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	986 # no important size here but a mode
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	987 if self._with_metadata_mtime:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	988 mtime = datetime.datetime.utcfromtimestamp(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	989 int(fso.stat.st_mtime))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	990 mtime = util.b(mtime.isoformat("T") + "Z")
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	991 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	992 b"5:mtime,%d:%s,", len(mtime), mtime))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	993 if self._with_metadata_full_mode:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	994 modestr = util.b(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	995 normalized_mode_str(fso.stat.st_mode))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	996 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	997 b"8:fullmode,%d:%s,", len(modestr), modestr))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	998 elif self._with_metadata_mode:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	999 modestr = util.b(normalized_compatible_mode_str(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1000 fso.stat.st_mode))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1001 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1002 b"4:mode,%d:%s,", len(modestr), modestr))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1003 if self._size_only:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1004 self._writer.write_size(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1005 util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1006 b"%s/./%s", opath, special_tag),
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1007 -1)
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1008 else:
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1009 sz = -1 if self._print_size else None
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1010 self._writer.write_file_digest(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1011 self._algorithm[1],
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1012 util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1013 b"%s/./%s", opath, special_tag),
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1014 b"",
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	1015 sz)
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1016 elif fso.is_dir:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1017 assert fso.stat is not None # because .is_dir is True
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1018 if fso.is_symlink and not self._follow_symlinks.directory:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1019 linktgt = walk.WalkDirEntry.from_readlink(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1020 os.readlink(fso.path))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1021 linkdgst = self._algorithm[0]()
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1022 if self._utf8_mode:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1023 if linktgt.u8path is None:
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1024 dir_tainted = True
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1025 linkdgst.update(linktgt.alt_u8path)
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1026 else:
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1027 linkdgst.update(linktgt.u8path)
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1028 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1029 if linktgt.fspath is None:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1030 dir_tainted = True
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1031 linkdgst.update(linktgt.alt_fspath)
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1032 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1033 linkdgst.update(linktgt.fspath)
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1034 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1035 b"2:@/,%d:%s,",
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1036 len(effective_fso_name),
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1037 effective_fso_name))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1038 #
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1039 # - no mtime and no mode for symlinks
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1040 # - also does not count for dir_size
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1041 #
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1042 dir_dgst.update(util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1043 b"%d:%s,%d:%s,",
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1044 len(self._algorithm[1]), util.b(self._algorithm[1]),
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1045 len(linkdgst.digest()), linkdgst.digest()))
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1046 if self._size_only:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1047 self._writer.write_size(
218 dee891ed2307 FIX: Output of symlinks was not converted property but was written in bytes repr Franz Glasner <fzglas.hg@dom66.de> parents: 217 diff changeset	1048 util.interpolate_bytes(b"%s/./@/", opath),
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1049 -1)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1050 else:
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1051 sz = -1 if self._print_size else None
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1052 self._writer.write_file_digest(
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1053 self._algorithm[1],
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1054 util.interpolate_bytes(b"%s/./@/", opath),
334 5afece258bf2 treesum: FIX: add missing size param Franz Glasner <fzglas.hg@dom66.de> parents: 333 diff changeset	1055 linkdgst.digest(),
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	1056 sz)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1057 else:
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1058 #
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1059 # Follow the symlink to dir or handle a "real" directory
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1060 #
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1061
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1062 # Get subdir data from recursing into it
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1063 sub_dir_errno, sub_dir_algo, sub_dir_dgst, sub_dir_size = \
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1064 self._generate(root, top + (fso.name, ))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1065
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1066 #
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1067 # Check first whether the directory was selected to be
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1068 # excluded
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1069 #
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1070 if sub_dir_errno is None:
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1071 # Yes -- skipped
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1072 continue
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1073 if sub_dir_errno == 0:
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1074 if sub_dir_size is None:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1075 if self._print_size or self._size_only:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1076 dir_tainted = True
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1077 else:
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	1078 dir_size += (sub_dir_size or 0)
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1079 else:
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1080 dir_tainted = True
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1081 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1082 b"1:/,%d:%s,",
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1083 len(effective_fso_name),
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1084 effective_fso_name))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1085 if self._with_metadata_full_mode:
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1086 modestr = util.b(normalized_mode_str(fso.stat.st_mode))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1087 dir_dgst.update(util.interpolate_bytes(
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1088 b"8:fullmode,%d:%s,", len(modestr), modestr))
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1089 elif self._with_metadata_mode:
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1090 modestr = util.b(normalized_compatible_mode_str(
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1091 fso.stat.st_mode))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1092 dir_dgst.update(util.interpolate_bytes(
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1093 b"4:mode,%d:%s,", len(modestr), modestr))
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1094 if sub_dir_errno == 0:
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1095 dir_dgst.update(util.interpolate_bytes(
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1096 b"%d:%s,%d:%s,",
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1097 len(sub_dir_algo), util.b(sub_dir_algo),
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1098 len(sub_dir_dgst), sub_dir_dgst))
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1099 else:
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1100 # NOTE: error message is already printed here
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1101 dir_dgst.update(util.interpolate_bytes(
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1102 b"5:errno,%d:%s",
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1103 len(str(sub_dir_errno)),
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1104 util.b(str(sub_dir_errno))))
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1105 else:
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1106 if fso.is_symlink and not self._follow_symlinks.file:
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1107 #
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1108 # Symbolic link to some filesystem object which is not
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1109 # determined to be a link to a directory or some other
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1110 # special file (socket, FIFO, et al.).
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1111 #
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1112 linktgt = walk.WalkDirEntry.from_readlink(
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1113 os.readlink(fso.path))
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1114 linkdgst = self._algorithm[0]()
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1115 if self._utf8_mode:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1116 if linktgt.u8path is None:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1117 dir_tainted = True
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1118 linkdgst.update(linktgt.alt_u8path)
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1119 else:
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1120 linkdgst.update(linktgt.u8path)
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1121 else:
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1122 if linktgt.fspath is None:
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1123 dir_tainted = True
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1124 linkdgst.update(linktgt.alt_fspath)
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1125 else:
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1126 linkdgst.update(linktgt.fspath)
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1127 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1128 b"1:@,%d:%s,",
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1129 len(effective_fso_name),
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1130 effective_fso_name))
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1131 #
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1132 # - no mtime and no mode for symlinks
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1133 # - also does not count for dir_size
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1134 #
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1135 dir_dgst.update(util.interpolate_bytes(
270 42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1136 b"%d:%s,%d:%s,",
42f4ca423ab3 treesum: REFACTOR: Major refactoring of computing digests: Franz Glasner <fzglas.hg@dom66.de> parents: 269 diff changeset	1137 len(self._algorithm[1]), util.b(self._algorithm[1]),
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1138 len(linkdgst.digest()), linkdgst.digest()))
8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1139 if self._size_only:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1140 self._writer.write_size(
218 dee891ed2307 FIX: Output of symlinks was not converted property but was written in bytes repr Franz Glasner <fzglas.hg@dom66.de> parents: 217 diff changeset	1141 util.interpolate_bytes(b"%s/./@", opath),
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1142 -1)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1143 else:
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1144 sz = -1 if self._print_size else None
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1145 self._writer.write_file_digest(
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1146 self._algorithm[1],
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1147 util.interpolate_bytes(b"%s/./@", opath),
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1148 linkdgst.digest(),
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	1149 sz)
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1150 else:
223 61e5b1c2685c Commentx Franz Glasner <fzglas.hg@dom66.de> parents: 221 diff changeset	1151 #
61e5b1c2685c Commentx Franz Glasner <fzglas.hg@dom66.de> parents: 221 diff changeset	1152 # Follow the symlink to file or handle a "real" file
61e5b1c2685c Commentx Franz Glasner <fzglas.hg@dom66.de> parents: 221 diff changeset	1153 #
61e5b1c2685c Commentx Franz Glasner <fzglas.hg@dom66.de> parents: 221 diff changeset	1154
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1155 dir_dgst.update(util.interpolate_bytes(
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1156 b"0:,%d:%s,",
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1157 len(effective_fso_name),
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1158 effective_fso_name))
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1159 if fso.stat is None:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1160 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1161 # Error: most likely a broken symlink here
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1162 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1163 dir_tainted = True
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1164 dir_dgst.update(util.interpolate_bytes(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1165 b"5:errno,%d:%s,",
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1166 len(str(fso.stat_errno)),
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1167 util.b(str(fso.stat_errno))))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1168 self._writer.write_error(util.interpolate_bytes(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1169 b"errno %d: %s",
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1170 fso.stat_errno,
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1171 util.b(fso.stat_errstr, "utf-8")))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1172 logging.error(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1173 "Directory entry has symlink problems: %r",
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1174 opath)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1175 if self._size_only:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1176 self._writer.write_size(opath, None)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1177 else:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1178 self._writer.write_file_digest(
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	1179 self._algorithm[1], opath, None, None)
217 8e38c07c4b85 Handle symlinks to files fully and Implement no-follow-file-symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 213 diff changeset	1180 else:
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1181 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1182 # Ok: File has normal stat info
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1183 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1184 # XXX FIXME: Handle special files (fifo, socket,
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1185 # block or char devices, ...).
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1186 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1187 dir_size += fso.stat.st_size
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1188 if self._with_metadata_mtime:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1189 mtime = datetime.datetime.utcfromtimestamp(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1190 int(fso.stat.st_mtime))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1191 mtime = util.b(mtime.isoformat("T") + "Z")
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1192 dir_dgst.update(util.interpolate_bytes(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1193 b"5:mtime,%d:%s,", len(mtime), mtime))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1194 if self._with_metadata_full_mode:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1195 modestr = util.b(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1196 normalized_mode_str(fso.stat.st_mode))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1197 dir_dgst.update(util.interpolate_bytes(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1198 b"8:fullmode,%d:%s,", len(modestr), modestr))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1199 elif self._with_metadata_mode:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1200 modestr = util.b(normalized_compatible_mode_str(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1201 fso.stat.st_mode))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1202 dir_dgst.update(util.interpolate_bytes(
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1203 b"4:mode,%d:%s,", len(modestr), modestr))
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1204 if self._size_only:
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1205 #
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1206 # size can be printed here because .stat is
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1207 # available
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1208 #
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1209 self._writer.write_size(opath, fso.stat.st_size)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1210 else:
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1211 try:
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1212 dgst = digest.compute_digest_file(
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1213 self._algorithm[0],
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1214 fso.path,
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1215 use_mmap=self._use_mmap)
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1216 except OSError as e:
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1217 dir_tainted = True
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1218 self._writer.write_error(
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1219 util.interpolate_bytes(
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1220 b"`%s': errno %d: %s",
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1221 opath,
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1222 e.errno,
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1223 util.b(e.strerror, "utf-8")))
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1224 sz = (fso.stat.st_size if self._print_size
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1225 else None)
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1226 self._writer.write_file_digest(
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	1227 self._algorithm[1], opath, None, sz)
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1228 else:
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1229 dir_dgst.update(util.interpolate_bytes(
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1230 b"%d:%s,%d:%s,",
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1231 len(self._algorithm[1]),
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1232 util.b(self._algorithm[1]),
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1233 len(dgst),
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1234 dgst))
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1235 sz = (fso.stat.st_size if self._print_size
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1236 else None)
9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1237 self._writer.write_file_digest(
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	1238 self._algorithm[1], opath, dgst, sz)
267 b9aa65a30b4c treesum: optimize the use of flush() somewhat Franz Glasner <fzglas.hg@dom66.de> parents: 266 diff changeset	1239 self._writer.flush()
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1240 opath = join_output_path(top, None)
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1241 if opath:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1242 if self._utf8_mode:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1243 opath = walk.WalkDirEntry.alt_u8(opath)
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1244 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1245 opath = walk.WalkDirEntry.alt_fs(opath)
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1246 if dir_tainted:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1247 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1248 # IMPORTANT: Print errors BEFORE the associated digest or size
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1249 # line. Otherwise the "info" command has a problem.
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1250 #
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1251 self._writer.write_error(b"directory is tainted")
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1252 logging.error("Directory has problems: %r", opath)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1253 if self._size_only:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1254 self._writer.write_size(opath, dir_size)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1255 else:
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1256 sz = dir_size if self._print_size else None
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1257 self._writer.write_file_digest(
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	1258 self._algorithm[1], opath, dir_dgst.digest(), sz)
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1259 self._writer.flush()
277 9676ecd32a07 treesum: FIX: Proper error handling. Franz Glasner <fzglas.hg@dom66.de> parents: 275 diff changeset	1260 return (0, self._algorithm[1], dir_dgst.digest(), dir_size)
177 089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1261
089c40240061 Add an alternate implementation for generating directory tree digests: Franz Glasner <fzglas.hg@dom66.de> parents: 176 diff changeset	1262
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1263 def join_output_path(top, name):
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1264 if name is None:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1265 # a path for a directory is to be computed
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1266 if top:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1267 if isinstance(top[0], bytes):
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1268 return b"/".join(top) + b"/"
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1269 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1270 return u"/".join(top) + u"/"
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1271 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1272 return b""
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1273 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1274 # a path for a normal file is to be computed
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1275 if top:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1276 if isinstance(name, bytes):
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1277 return b"/".join(top) + b"/" + name
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1278 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1279 return u"/".join(top) + u"/" + name
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1280 else:
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1281 return name
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1282
07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1283
158 d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	1284 def normalized_compatible_mode_str(mode):
157 27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	1285 # XXX FIXME: Windows and "executable"
27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	1286 modebits = stat.S_IMODE(mode)
27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	1287 modestr = "%o" % (modebits,)
27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	1288 if not modestr.startswith("0"):
27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	1289 modestr = "0" + modestr
27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	1290 return modestr
27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	1291
27d1aaf5fe39 Implement "--mode" flag for "treesum.py" to consider file portable mode bits Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	1292
158 d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	1293 def normalized_mode_str(mode):
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	1294 modestr = "%o" % (mode,)
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	1295 if not modestr.startswith("0"):
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	1296 modestr = "0" + modestr
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	1297 return modestr
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	1298
d8cdd1985d43 Implement "--full-mode" for "treesum.py" Franz Glasner <fzglas.hg@dom66.de> parents: 157 diff changeset	1299
325 cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1300 class WriterBase(object):
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1301
336 0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1302 """(Abstract) base class for all treesum digest file writers.
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1303
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1304 Wraps an output file pointer for a binary file.
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1305
325 cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1306 Provides low-level methods to write data lines. These methods must be
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1307 used if the CRC is to be updated also.
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1308
333 2a244a05bbb5 treesum: FIX: docs Franz Glasner <fzglas.hg@dom66.de> parents: 332 diff changeset	1309 Also holds some very common attributes that control some aspects
2a244a05bbb5 treesum: FIX: docs Franz Glasner <fzglas.hg@dom66.de> parents: 332 diff changeset	1310 of the output format (e.g. `.LS`, `.use_base64`).
2a244a05bbb5 treesum: FIX: docs Franz Glasner <fzglas.hg@dom66.de> parents: 332 diff changeset	1311
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1312 Also holds the current CRC for a block.
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1313
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1314 """
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1315
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1316 LS = util.b(os.linesep)
332 5b98810fe367 treesum: attribute comment for "LS" Franz Glasner <fzglas.hg@dom66.de> parents: 331 diff changeset	1317 """Because we write the output as binary files we need the official line
5b98810fe367 treesum: attribute comment for "LS" Franz Glasner <fzglas.hg@dom66.de> parents: 331 diff changeset	1318 separator for you OS as bytes"""
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1319
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1320 DEFAULT_GROUPING_SEPARATOR = ""
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1321 """Disable the thousands separator in case no subclass redefines it"""
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1322
336 0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1323 def __init__(self, outfp, size_only=False, print_size=False,
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1324 use_base64=False, grouping_separator=None,
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1325 size_column_width=None):
336 0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1326 # Poor man's abstract abstract class implemenetation
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1327 assert self.__class__ is not WriterBase
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1328
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1329 self._outfp = outfp
336 0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1330 self.size_only = size_only
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1331 self.print_size = print_size
331 9ee84624587f treesum: move the handling of base64 digest output into the writers completely Franz Glasner <fzglas.hg@dom66.de> parents: 330 diff changeset	1332 self.use_base64 = use_base64
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1333 self.grouping_separator = (grouping_separator
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1334 if grouping_separator is not None
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1335 else self.DEFAULT_GROUPING_SEPARATOR)
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1336 self.size_column_width = size_column_width or 0
325 cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1337 self.reset_crc()
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1338
336 0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1339 def write_size(self, filename, sz):
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1340 """
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1341
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1342 If `sz` is `None` then this is signals an error because a size is
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1343 required.
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1344
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1345 If the size should not be printed on purpose the `size` should be
e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1346 as negative number.
e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1347
336 0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1348 """
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1349 raise NotImplementedError("write_size")
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1350
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	1351 def write_file_digest(self, algorithm, filename, digest, size):
336 0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1352 """
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1353
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1354 If `size` is `None` and the output of a size is required then this
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1355 is an error signal.
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1356
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1357 If the size should not be printed on purpose the `size` should be
e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1358 as negative number.
e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1359
e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1360 If `digest` is `None` is an error signal.
e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1361
336 0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1362 """
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1363 raise NotImplementedError("write_file_digest")
0049f486e1cd treesum: Also provide the WriterBase with flags when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 335 diff changeset	1364
325 cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1365 @property
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1366 def crc(self):
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1367 return self._crc
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1368
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1369 def reset_crc(self):
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1370 self._crc = crc32()
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1371
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1372 def writeln(self, line):
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1373 """Write the bytes `line` into the output file and update the CRC
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1374 accordingly.
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1375
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1376 :param bytes line: The line to write to (without line ending)
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1377
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1378 """
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1379 self.write(line)
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1380 self.write(self.LS)
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1381
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1382 def write(self, data):
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1383 """Write `data` into the output file and update the CRC accordingly.
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1384
325 cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1385 :param bytes data: The data to write to and to update the CRC with
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1386
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1387 """
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1388 if data:
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1389 self._outfp.write(data)
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1390 self._crc.update(data)
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1391
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1392 def flush(self):
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1393 self._outfp.flush()
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1394
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1395
326 a464f36fffcb treesum: Rename the current TreesumWriter to TaggedTreesumWriter. Franz Glasner <fzglas.hg@dom66.de> parents: 325 diff changeset	1396 class TaggedTreesumWriter(WriterBase):
325 cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1397
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1398 """Writer to write treesum digest files in a format similar to BSD
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1399 digest files.
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1400
333 2a244a05bbb5 treesum: FIX: docs Franz Glasner <fzglas.hg@dom66.de> parents: 332 diff changeset	1401 Provides high-level methods to write data lines.
325 cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1402
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1403 """
cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1404
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1405 DEFAULT_GROUPING_SEPARATOR = '_'
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1406 """The default thousands separator"""
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1407
325 cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1408 def __init__(self, outfp, **kwds):
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1409 # No alignment for the size here
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1410 kwds["size_column_width"] = 0
331 9ee84624587f treesum: move the handling of base64 digest output into the writers completely Franz Glasner <fzglas.hg@dom66.de> parents: 330 diff changeset	1411 super(TaggedTreesumWriter, self).__init__(outfp, **kwds)
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1412
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1413 def start(self, version):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1414 """Begin a new block, reset the current CRC and write the VERSION
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1415 tag.
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1416
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1417 """
325 cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1418 self.reset_crc()
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1419 self.write(b"VERSION = ")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1420 self.writeln(util.b(version))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1421
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1422 def write_comment(self, comment):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1423 self.write(b"COMMENT (")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1424 self.write(util.b(comment, "utf-8"))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1425 self.writeln(b")")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1426
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1427 def write_generator(self, generator):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1428 self.write(b"GENERATOR (")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1429 self.write(util.b(generator, "utf-8"))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1430 self.writeln(b")")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1431
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1432 def write_error(self, error):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1433 self.write(b"ERROR (")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1434 self.write(util.b(error, "utf-8"))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1435 self.writeln(b")")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1436
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1437 def write_fsencoding(self, encoding):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1438 self.write(b"FSENCODING = ")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1439 self.writeln(util.b(encoding))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1440
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1441 def write_fnmatch_pattern(self, action, kind, pattern):
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1442 self.write(b"FNMATCH (")
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1443 self.write(util.b(action))
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1444 self.write(b": ")
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1445 self.write(util.b(kind))
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1446 self.write(b":")
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1447 self.write(util.b(pattern, "utf-8"))
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1448 self.writeln(b")")
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1449
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1450 def write_flags(self, flags):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1451 self.write(b"FLAGS = ")
262 c3d6599c1b5e FIX: treesum: bytearray needs not to be handled Franz Glasner <fzglas.hg@dom66.de> parents: 261 diff changeset	1452 if isinstance(flags, (str, bytes)):
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1453 self.writeln(util.b(flags))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1454 else:
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1455 flags.sort()
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1456 self.writeln(util.b(",".join(flags)))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1457
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1458 def write_timestamp(self, ts):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1459 self.write(b"TIMESTAMP = ")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1460 self.writeln(util.b(str(ts)))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1461
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1462 def write_isotimestamp(self, ts):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1463 self.write(b"ISOTIMESTAMP = ")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1464 self.writeln(util.b(ts))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1465
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1466 def write_root(self, root):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1467 assert isinstance(root, bytes)
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1468 self.write(b"ROOT (")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1469 self.write(root)
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1470 self.writeln(b")")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1471
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1472 def write_size(self, filename, sz):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1473 assert isinstance(filename, bytes)
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1474 self.write(b"SIZE (")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1475 self.write(filename)
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1476 self.write(b")")
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1477 if sz is not None:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1478 self.write(b" = ")
339 9c7a03199092 treesum: Use the grouping separator (aka thousands separator) when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 338 diff changeset	1479 self.write(
9c7a03199092 treesum: Use the grouping separator (aka thousands separator) when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 338 diff changeset	1480 b"" if sz < 0
9c7a03199092 treesum: Use the grouping separator (aka thousands separator) when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 338 diff changeset	1481 else util.b(format(sz, ',').replace(
9c7a03199092 treesum: Use the grouping separator (aka thousands separator) when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 338 diff changeset	1482 ',', self.grouping_separator)))
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1483 self.writeln(b"")
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1484
317 fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1485 def write_accept_treesum_file(self, filename):
fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1486 assert isinstance(filename, bytes)
fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1487 self.write(b"ACCEPT-TREESUM (")
fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1488 self.write(filename)
fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1489 self.writeln(b")")
fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1490
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	1491 def write_file_digest(self, algorithm, filename, digest, size):
314 1fbe9904b188 treesum: assert also in .write_accept_treesum() that the given filename is of bytes type Franz Glasner <fzglas.hg@dom66.de> parents: 313 diff changeset	1492 assert isinstance(filename, bytes)
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1493 if digest is not None:
341 728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1494 if digest != b"":
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1495 digest = (base64.b64encode(digest)
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1496 if self.use_base64
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1497 else binascii.hexlify(digest))
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1498 self.write(util.b(algorithm))
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1499 self.write(b" (")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1500 self.write(filename)
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1501 self.write(b")")
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1502 if digest is not None or size is not None:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1503 self.write(b" = ")
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1504 if digest is not None:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1505 self.write(digest)
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1506 if size is not None:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1507 self.write(b",")
339 9c7a03199092 treesum: Use the grouping separator (aka thousands separator) when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 338 diff changeset	1508 self.write(
9c7a03199092 treesum: Use the grouping separator (aka thousands separator) when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 338 diff changeset	1509 b"" if size < 0
9c7a03199092 treesum: Use the grouping separator (aka thousands separator) when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 338 diff changeset	1510 else util.b(format(size, ',').replace(
9c7a03199092 treesum: Use the grouping separator (aka thousands separator) when printing sizes Franz Glasner <fzglas.hg@dom66.de> parents: 338 diff changeset	1511 ',', self.grouping_separator)))
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1512 self.writeln(b"")
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1513
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1514 def finish(self):
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1515 """Finish a block and write the current CRC"""
325 cd458e318553 treesum: break up TreesumWriter in TreesumWriter and a new base class WriterBase. Franz Glasner <fzglas.hg@dom66.de> parents: 323 diff changeset	1516 crc = self.crc.hexdigest()
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1517 self.write(b"CRC32 = ")
a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1518 self.writeln(util.b(crc))
267 b9aa65a30b4c treesum: optimize the use of flush() somewhat Franz Glasner <fzglas.hg@dom66.de> parents: 266 diff changeset	1519 self.flush()
261 a3e25957afb7 treesum: instead of using format_bsd_line use a real write object with specialized methods Franz Glasner <fzglas.hg@dom66.de> parents: 260 diff changeset	1520
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	1521
330 61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1522 class TabularTreesumWriter(WriterBase):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1523
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1524 """Writer to write treesum digest files in a format similar to tabular
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1525 GNU digest files.
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1526
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1527 Provides high-level methods to write data lines.
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1528
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1529 """
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1530
335 7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1531 DEFAULT_GROUPING_SEPARATOR = '.'
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1532 """The default thousands separator"""
7620142aacd1 treesum: prepare for a grouping-separator (aka thousands separator) for sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 334 diff changeset	1533
330 61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1534 def __init__(self, outfp, **kwds):
331 9ee84624587f treesum: move the handling of base64 digest output into the writers completely Franz Glasner <fzglas.hg@dom66.de> parents: 330 diff changeset	1535 super(TabularTreesumWriter, self).__init__(outfp, **kwds)
341 728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1536 # Prepare some format strings for performance reasons
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1537 if self.size_column_width > 0:
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1538 self._formatstring_size = '>' + str(self.size_column_width)
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1539 self._errorstring_size = b'?' * self.size_column_width
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1540 self._emptystring_size = b' ' * self.size_column_width
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1541 else:
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1542 self._formatstring_size = ">"
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1543 self._errorstring_size = b"?????"
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1544 self._emptystring_size = b''
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1545 if self.grouping_separator:
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1546 self._formatstring_size += ','
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1547 self._formatstring_size += 'd'
330 61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1548
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1549 def start(self, version):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1550 """Begin a new block, reset the current CRC and write the VERSION
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1551 tag.
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1552
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1553 """
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1554 self.reset_crc()
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1555 self.write(b"VERSION\t")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1556 self.writeln(util.b(version))
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1557
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1558 def write_comment(self, comment):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1559 self.write(b"COMMENT\t")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1560 self.writeln(util.b(comment, "utf-8"))
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1561
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1562 def write_generator(self, generator):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1563 self.write(b"GENERATOR\t")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1564 self.writeln(util.b(generator, "utf-8"))
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1565
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1566 def write_error(self, error):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1567 self.write(b"ERROR\t")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1568 self.writeln(util.b(error, "utf-8"))
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1569
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1570 def write_fsencoding(self, encoding):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1571 self.write(b"FSENCODING\t")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1572 self.writeln(util.b(encoding))
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1573
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1574 def write_fnmatch_pattern(self, action, kind, pattern):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1575 self.write(b"FNMATCH\t")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1576 self.write(util.b(action))
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1577 self.write(b": ")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1578 self.write(util.b(kind))
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1579 self.write(b":")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1580 self.writeln(util.b(pattern, "utf-8"))
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1581
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1582 def write_flags(self, flags):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1583 self.write(b"FLAGS\t")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1584 if isinstance(flags, (str, bytes)):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1585 self.writeln(util.b(flags))
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1586 else:
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1587 flags.sort()
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1588 self.writeln(util.b(",".join(flags)))
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1589
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1590 def write_timestamp(self, ts):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1591 self.write(b"TIMESTAMP\t")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1592 self.writeln(util.b(str(ts)))
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1593
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1594 def write_isotimestamp(self, ts):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1595 self.write(b"ISOTIMESTAMP\t")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1596 self.writeln(util.b(ts))
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1597
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1598 def write_root(self, root):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1599 assert isinstance(root, bytes)
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1600 self.write(b"ROOT\t")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1601 self.writeln(root)
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1602
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1603 def write_size(self, filename, sz):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1604 assert isinstance(filename, bytes)
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1605 if sz is not None:
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1606 if sz >= 0:
341 728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1607 self.write(util.b(format(
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1608 sz, self._formatstring_size).replace(
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1609 ',', self.grouping_separator)))
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1610 else:
341 728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1611 self.write(self._emptystring_size)
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1612 else:
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1613 self.write(self._errorstring_size)
330 61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1614 self.write(b"\t")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1615 self.writeln(filename)
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1616
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1617 def write_accept_treesum_file(self, filename):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1618 assert isinstance(filename, bytes)
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1619 self.write(b"ACCEPT-TREESUM\t")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1620 self.writeln(filename)
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1621
337 e89eb63fc319 treesum: .write_file_digest() method: the "size" parameter is now positional and mandatory Franz Glasner <fzglas.hg@dom66.de> parents: 336 diff changeset	1622 def write_file_digest(self, algorithm, filename, digest, size):
330 61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1623 assert isinstance(filename, bytes)
341 728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1624 if digest is not None and digest != b"":
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1625 digest = (base64.b64encode(digest) if self.use_base64
330 61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1626 else binascii.hexlify(digest))
341 728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1627 else:
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1628 #
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1629 # Compute an error digest string with the "correct" length for
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1630 # given algorithm
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1631 #
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1632 nulldigest = b'\0' * util.algotag2digest_size(algorithm)
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1633 dsz = len(base64.b64encode(nulldigest) if self.use_base64
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1634 else binascii.hexlify(nulldigest))
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1635 if digest is None:
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1636 digest = b'?' * dsz
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1637 else:
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1638 digest = b' ' * dsz
330 61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1639 self.write(util.b(algorithm))
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1640 self.write(b":")
341 728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1641 self.write(digest)
330 61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1642 self.write(b"\t")
341 728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1643 if self.print_size:
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1644 if size is not None:
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1645 if size >= 0:
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1646 self.write(util.b(format(
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1647 size, self._formatstring_size).replace(
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1648 ',', self.grouping_separator)))
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1649 else:
728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1650 self.write(self._emptystring_size)
338 e163e6754071 treesum: when printing an empty size on purpose do not provide an empty octet string but use -1 as size argument Franz Glasner <fzglas.hg@dom66.de> parents: 337 diff changeset	1651 else:
341 728ad9c639f2 treesum: right-align the output of size when using the tabular style Franz Glasner <fzglas.hg@dom66.de> parents: 340 diff changeset	1652 self.write(self._errorstring_size)
330 61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1653 self.write(b"\t")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1654 self.writeln(filename)
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1655
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1656 def finish(self):
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1657 """Finish a block and write the current CRC"""
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1658 crc = self.crc.hexdigest()
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1659 self.write(b"CRC32\t")
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1660 self.writeln(util.b(crc))
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1661 self.flush()
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1662
61cbae10103c treesum: first minimal version of writing the treesum output in tabular format. Franz Glasner <fzglas.hg@dom66.de> parents: 329 diff changeset	1663
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1664 class TreesumReader(object):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1665
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1666 """Reader to read and/or verify treesum digest files.
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1667
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1668 Supports the iterator and context manager protocol.
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1669
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1670 """
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1671
204 07f1d79e6674 Fully implemented UTF-8 mode for treeview. Franz Glasner <fzglas.hg@dom66.de> parents: 202 diff changeset	1672 PATTERN0 = re.compile(br"\A[ \t]*\r?\n\Z") # empty lines
198 c1e875ba4bdc Put the effective filesystem encoding into the treesum digest file using FSENCODING = <encoding> Franz Glasner <fzglas.hg@dom66.de> parents: 197 diff changeset	1673 PATTERN1 = re.compile(br"\A(VERSION\|FSENCODING\|FLAGS\|TIMESTAMP\|ISOTIMESTAMP\|CRC32)[ \t]=[ \t]([^ \t]+)[ \t]*\r?\n\Z") # noqa: E501 line too long
317 fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1674 PATTERN2 = re.compile(br"\A(ROOT\|COMMENT\|ERROR\|GENERATOR\|FNMATCH\|ACCEPT-TREESUM)[ \t]\((.)\)[ \t]*\r?\n\Z") # noqa: E501 line too long
340 cfa544fbb9f9 treesum: allow parsing of files again where sizes are printed with grouping with standard separators out of "., '_" Franz Glasner <fzglas.hg@dom66.de> parents: 339 diff changeset	1675 PATTERN3 = re.compile(br"\ASIZE[ \t]\((.)\)([ \t]=[ \t]([0-9., '_][0-9]))?[ \t]\r?\n\Z") # noqa: E501 line too long
cfa544fbb9f9 treesum: allow parsing of files again where sizes are printed with grouping with standard separators out of "., '_" Franz Glasner <fzglas.hg@dom66.de> parents: 339 diff changeset	1676 PATTERN4 = re.compile(br"\A([A-Za-z0-9_-]+)[ \t]\((.)\)([ \t]=[ \t]([A-Za-z0-9=+/]+)?(,([0-9., '_][0-9])?)?)?[ \t]\r?\n\Z") # noqa: E501 line too long
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1677
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1678 def __init__(self, _fp, _filename, _own_fp):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1679 self._fp = _fp
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1680 self._own_fp = _own_fp
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1681 self._filename = _filename
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1682 self._line_no = 0
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1683 self._reset_crc()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1684 self._expect_crc = None # NOTE: tristate: None is different from False
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1685 self._current_algo_name = self._current_algo_digest_size = None
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1686
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1687 @classmethod
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1688 def from_path(cls_, path):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1689 """Open file at `path` and return a reader that owns the file object"""
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1690 return cls_(open(path, "rb"), path, True)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1691
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1692 @classmethod
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1693 def from_binary_buffer(cls_, binary_fp, filename):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1694 return cls_(binary_fp, filename, False)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1695
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1696 def __enter__(self):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1697 return self
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1698
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1699 def __exit__(self, *args):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1700 self.close()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1701
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1702 def close(self):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1703 if self._fp is not None:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1704 try:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1705 if self._own_fp:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1706 self._fp.close()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1707 finally:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1708 self._fp = None
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1709
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1710 def __iter__(self):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1711 return self
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1712
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1713 def __next__(self):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1714 rec = self.read_record()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1715 if rec is None:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1716 raise StopIteration()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1717 return rec
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1718
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1719 if util.PY2:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1720 next = __next__
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1721
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1722 def all_records(self):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1723 """Iterator over all remaining records"""
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1724 while True:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1725 rec = self.read_record()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1726 if rec is None:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1727 return
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1728 yield rec
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1729
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1730 def read_record(self):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1731 """Read and parse the "next" line.
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1732
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1733 :returns: `None` at EOF or the parsed contents of the line
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1734 :rtype: tuple or None
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1735
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1736 """
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1737 # Loop to skip empty lines
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1738 while True:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1739 line = self._get_next_line()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1740 if not line:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1741 #
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1742 # Skip for empty files at the very beginning.
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1743 # Check only after the first VERSION line.
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1744 #
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1745 if self._expect_crc is not None:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1746 if self._expect_crc:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1747 logging.warning("CRC32 is missing at EOF")
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1748 return None
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1749 if not self.PATTERN0.search(line):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1750 break
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1751 self._update_crc(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1752 #
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1753 # At the beginning transparently skip an eventually embedded signify
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1754 # signature
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1755 #
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1756 if self._line_no == 1:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1757 if line.startswith(b"untrusted comment: "):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1758 line = self._get_next_line()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1759 if not line.endswith(b"\n"):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1760 raise binascii.Error("No valid signify signature value")
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1761 # Try to decode for an early error check
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1762 base64.b64decode(line[:-1])
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1763 mo = self.PATTERN1.search(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1764 if mo:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1765 if mo.group(1) == b"VERSION":
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1766 if self._expect_crc:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1767 logging.warning("CRC32 missing before line %d",
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1768 self._line_no)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1769 self._reset_crc()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1770 self._expect_crc = True
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1771 self._update_crc(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1772 return ("VERSION", util.n(mo.group(2)))
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1773 if mo.group(1) == b"CRC32":
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1774 # TODO: check
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1775 if self._expect_crc is None:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1776 logging.warning("Lone CRC32 before VERSION in line %d",
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1777 self._line_no)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1778 else:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1779 if self._expect_crc:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1780 if (self._hex_crc()
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1781 != mo.group(2).decode("latin1").upper()):
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1782 logging.warning(
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1783 "CRC32 mismatch in line %d:"
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1784 " expected: %s, given: %s",
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1785 self._line_no,
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1786 self._hex_crc(),
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1787 mo.group(2).decode("latin1").upper())
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1788 else:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1789 logging.warning("CRC32 before VERSION in line %d",
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1790 self._line_no)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1791 # Do not update the CRC here but reset the state
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1792 self._expect_crc = False
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1793 return ("CRC32", util.n(mo.group(2)))
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1794 else:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1795 self._update_crc(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1796 return (util.n(mo.group(1)), util.n(mo.group(2)))
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1797 else:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1798 mo = self.PATTERN2.search(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1799 if mo:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1800 self._update_crc(line)
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1801 if mo.group(1) in (b"COMMENT", b"ERROR", b"GENERATOR",
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1802 b"FNMATCH"):
316 3f1f1c2e9e5f treesum: FIX: returned tag should be a native string Franz Glasner <fzglas.hg@dom66.de> parents: 315 diff changeset	1803 return (util.n(mo.group(1)), util.u(mo.group(2), "utf-8"))
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1804 elif mo.group(1) == b"ROOT":
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1805 return ("ROOT", mo.group(2))
317 fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1806 elif mo.group(1) == b"ACCEPT-TREESUM":
fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1807 return ("ACCEPT-TREESUM", mo.group(2))
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1808 assert False, line
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1809 else:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1810 mo = self.PATTERN3.search(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1811 if mo:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1812 self._update_crc(line)
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1813 if mo.group(2):
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1814 return ("SIZE", mo.group(1),
340 cfa544fbb9f9 treesum: allow parsing of files again where sizes are printed with grouping with standard separators out of "., '_" Franz Glasner <fzglas.hg@dom66.de> parents: 339 diff changeset	1815 int(util.n(mo.group(3))
cfa544fbb9f9 treesum: allow parsing of files again where sizes are printed with grouping with standard separators out of "., '_" Franz Glasner <fzglas.hg@dom66.de> parents: 339 diff changeset	1816 .translate(None, "., '_"),
cfa544fbb9f9 treesum: allow parsing of files again where sizes are printed with grouping with standard separators out of "., '_" Franz Glasner <fzglas.hg@dom66.de> parents: 339 diff changeset	1817 10))
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1818 else:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1819 return ("SIZE", mo.group(1), None)
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1820 else:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1821 mo = self.PATTERN4.search(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1822 if mo:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1823 self._update_crc(line)
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1824 algo_name = util.n(mo.group(1))
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1825 if mo.group(3):
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1826 if mo.group(4):
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1827 if (len(mo.group(4)) ==
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1828 2 * self._get_digest_size(algo_name)):
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1829 # hex
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1830 digest = binascii.unhexlify(mo.group(4))
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1831 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1832 # base64
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1833 digest = base64.b64decode(mo.group(4))
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1834 else:
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1835 digest = None
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1836 if mo.group(5):
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1837 if mo.group(6):
340 cfa544fbb9f9 treesum: allow parsing of files again where sizes are printed with grouping with standard separators out of "., '_" Franz Glasner <fzglas.hg@dom66.de> parents: 339 diff changeset	1838 size = int(util.n(mo.group(6))
cfa544fbb9f9 treesum: allow parsing of files again where sizes are printed with grouping with standard separators out of "., '_" Franz Glasner <fzglas.hg@dom66.de> parents: 339 diff changeset	1839 .translate(None, "., '_"),
cfa544fbb9f9 treesum: allow parsing of files again where sizes are printed with grouping with standard separators out of "., '_" Franz Glasner <fzglas.hg@dom66.de> parents: 339 diff changeset	1840 10)
275 c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1841 else:
c72f5b2dbc6f treesum: Simplify the path computations and make the visible output more consistent. Franz Glasner <fzglas.hg@dom66.de> parents: 271 diff changeset	1842 size = None
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1843 else:
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1844 size = None
0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1845 return (algo_name, mo.group(2), digest, size)
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1846 else:
266 0add8276e6b8 treesum: Handle errors like broken symlinks properly Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	1847 return (algo_name, mo.group(2), None, None)
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1848 else:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1849 assert False, line
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1850 return line
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1851
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1852 def _get_next_line(self):
189 959c6d37b014 Extend the max line length to read to 4096, which is something along PATH_MAX onn Linux Franz Glasner <fzglas.hg@dom66.de> parents: 188 diff changeset	1853 line = self._fp.readline(4096) # along PATH_MAX on Linux
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1854 if line:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1855 self._line_no += 1
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1856 return line
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1857
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1858 def _reset_crc(self):
260 07a0bc723139 treesum: Implement the CRC-32 using the new util.crc32 module Franz Glasner <fzglas.hg@dom66.de> parents: 223 diff changeset	1859 self._crc32 = crc32()
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1860
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1861 def _update_crc(self, data):
260 07a0bc723139 treesum: Implement the CRC-32 using the new util.crc32 module Franz Glasner <fzglas.hg@dom66.de> parents: 223 diff changeset	1862 self._crc32.update(data)
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1863
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1864 def _hex_crc(self):
260 07a0bc723139 treesum: Implement the CRC-32 using the new util.crc32 module Franz Glasner <fzglas.hg@dom66.de> parents: 223 diff changeset	1865 return self._crc32.hexdigest()
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1866
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1867 def _get_digest_size(self, algo_name):
307 64df94bf4659 treesum: Build a little static database of digest sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 306 diff changeset	1868 """Get the `digest_size` from algorithm specifier `algo_name`.
64df94bf4659 treesum: Build a little static database of digest sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 306 diff changeset	1869
64df94bf4659 treesum: Build a little static database of digest sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 306 diff changeset	1870 Cache this on the assumption, that algorithms do not change very
64df94bf4659 treesum: Build a little static database of digest sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 306 diff changeset	1871 often. Do this because the `digest_size` can only be given by a
64df94bf4659 treesum: Build a little static database of digest sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 306 diff changeset	1872 digest instance.
64df94bf4659 treesum: Build a little static database of digest sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 306 diff changeset	1873
64df94bf4659 treesum: Build a little static database of digest sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 306 diff changeset	1874 """
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1875 if self._current_algo_name == algo_name:
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1876 return self._current_algo_digest_size
307 64df94bf4659 treesum: Build a little static database of digest sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 306 diff changeset	1877 sz = util.algotag2digest_size(algo_name)
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1878 self._current_algo_name = algo_name
307 64df94bf4659 treesum: Build a little static database of digest sizes. Franz Glasner <fzglas.hg@dom66.de> parents: 306 diff changeset	1879 self._current_algo_digest_size = sz
188 2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1880 return self._current_algo_digest_size
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1881
2784fdcc99e5 Implement basic parsing of treesum output. Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	1882
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1883 def print_treesum_digestfile_infos(opts):
306 ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1884 get_infos_from_digestfile(
ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1885 opts.digest_files,
ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1886 print_block_data,
ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1887 opts.print_only_last_block)
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1888
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1889
306 ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1890 def get_infos_from_digestfile(digest_files, block_handler,
ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1891 only_last_block=True):
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1892 for fn in digest_files:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1893 if fn == "-":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1894 if util.PY2:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1895 reader = TreesumReader.from_binary_buffer(sys.stdin)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1896 else:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1897 reader = TreesumReader.from_binary_buffer(sys.stdin.buffer)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1898 else:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1899 reader = TreesumReader.from_path(fn)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1900
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1901 with reader:
208 85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1902 root = generator = flags = fsencoding = algorithm = digest \
85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1903 = size = None
219 19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1904 errors = set()
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1905 comments = []
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1906 fnmatch_filters = []
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1907 in_block = False
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1908 block_no = 0
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1909 for record in reader:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1910 if record[0] == "VERSION":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1911 assert record[1] == "1"
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1912 # start a new block
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1913 in_block = True
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1914 block_no += 1
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1915 root = flags = algorithm = digest = size = None
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1916 comments = []
208 85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1917 elif record[0] == "GENERATOR":
85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1918 generator = record[1]
85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1919 elif record[0] == "FSENCODING":
85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1920 fsencoding = record[1]
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1921 elif record[0] == "FLAGS":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1922 flags = record[1]
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1923 elif record[0] == "ROOT":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1924 root = record[1]
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1925 elif record[0] == "COMMENT":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1926 comments.append(record[1])
219 19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1927 elif record[0] == "ERROR":
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1928 errors.add(record[1])
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1929 elif record[0] == "FNMATCH":
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1930 fnmatch_filters.append(record[1])
208 85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1931 elif record[0] in ("TIMESTAMP", "ISOTIMESTAMP"):
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1932 pass
317 fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1933 elif record[0] == "ACCEPT-TREESUM":
fc1b940bd4a6 treesum: when accepting treesum digest files put a line with its filename into the output Franz Glasner <fzglas.hg@dom66.de> parents: 316 diff changeset	1934 pass
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1935 elif record[0] == "CRC32":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1936 pass
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1937 # in_block = False
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1938 else:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1939 if not in_block:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1940 continue
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1941 # digest line or size line
193 fb36e71f6ba8 Change: path indicators for symlinks: ./@ -> ./@/ and /./@ -> /./@/ Franz Glasner <fzglas.hg@dom66.de> parents: 191 diff changeset	1942 if not record[1] or record[1] == b"./@/":
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1943 if record[0] == "SIZE":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1944 algorithm = "SIZE"
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1945 digest = None
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1946 size = record[2]
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1947 else:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1948 algorithm = record[0]
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1949 digest = record[2]
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1950 size = record[3]
306 ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1951 if not only_last_block:
ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1952 block_handler(
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1953 block_no,
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1954 root, generator, fsencoding, flags,
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1955 fnmatch_filters,
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1956 comments, errors, algorithm, digest, size)
219 19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1957 root = generator = flags = fsencoding = algorithm \
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1958 = digest = size = None
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1959 errors = set()
19eaba51c632 Refactored the printing of FLAGS: print flags always and explicitely print symlink behaviour and encoding configuration Franz Glasner <fzglas.hg@dom66.de> parents: 218 diff changeset	1960 comments = []
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1961 in_block = False
306 ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1962 if only_last_block:
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1963 if not in_block:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1964 if digest is not None or size is not None:
306 ebddfdbc3f7a treesum: use a callback to print parsed .treesum output file infos Franz Glasner <fzglas.hg@dom66.de> parents: 303 diff changeset	1965 block_handler(
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1966 block_no,
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1967 root, generator, fsencoding, flags, fnmatch_filters,
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1968 comments, errors, algorithm, digest, size)
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1969 else:
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1970 logging.warning("missing block end")
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1971
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1972
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1973 def print_block_data(block_no, tag, generator, fsencoding, flags,
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1974 fnmatch_filters, comments, errors,
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1975 algorithm, digest, size):
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1976 digeststr = util.n(binascii.hexlify(digest)) if digest else "<no digest>"
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1977 sizestr = str(size) if size is not None else "<no size>"
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1978 print("BLOCK No %d:" % (block_no,))
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1979 print(" Tag:", tag)
208 85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1980 print(" FS-Encoding:", fsencoding)
85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1981 if generator:
85e7edea8ac7 Print the filesystem encoding and the generator in the "info" output also Franz Glasner <fzglas.hg@dom66.de> parents: 207 diff changeset	1982 print(" Generator:", generator)
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1983 print(" Flags:", flags if flags else "<none>")
220 8db78850d800 "treesum info": print comments only if not empty Franz Glasner <fzglas.hg@dom66.de> parents: 219 diff changeset	1984 if comments:
8db78850d800 "treesum info": print comments only if not empty Franz Glasner <fzglas.hg@dom66.de> parents: 219 diff changeset	1985 print(" Comments:", comments)
302 bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1986 if fnmatch_filters:
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1987 for f in fnmatch_filters:
bf88323d6bf7 treesum: Implement --exclude/--include. Franz Glasner <fzglas.hg@dom66.de> parents: 301 diff changeset	1988 print(" FNMatch:", f)
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1989 print(" Algorithm:", algorithm)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1990 if algorithm != "SIZE":
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1991 print(" Digest:", digeststr)
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	1992 print(" Size:", sizestr)
303 73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	1993 if errors:
73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	1994 errorlist = list(errors)
73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	1995 errorlist.sort()
73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	1996 for idx, err in enumerate(errorlist):
73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	1997 if idx == 0:
73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	1998 print(" Errors:", err)
73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	1999 else:
73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	2000 print(" ", err)
73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	2001 else:
73d13be531b5 treesum: in the "info" command print a sortes list of errors Franz Glasner <fzglas.hg@dom66.de> parents: 302 diff changeset	2002 print(" Errors: <none>")
190 7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	2003
7e0c25a31757 First implementation of "treeview info" to print some information from the treeview digest files Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	2004
308 652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2005 class DigestSizeCollector(object):
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2006
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2007 def __init__(self):
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2008 self._algorithm = self._digest = self._size = None
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2009
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2010 def __call__(self, block_no, tag, generator, fsencoding, flags,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2011 fnmatch_filters, comments, errors,
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2012 algorithm, digest, size):
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2013 self._algorithm = algorithm
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2014 self._digest = digest
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2015 self._size = size
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2016
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2017 @property
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2018 def algorithm(self):
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2019 return self._algorithm
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2020
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2021 @property
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2022 def digest(self):
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2023 return self._digest
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2024
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2025 @property
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2026 def size(self):
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2027 return self._size
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2028
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2029 def collect_from_file(self, digest_file):
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2030 get_infos_from_digestfile([digest_file], self, True)
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2031
652870b20f9e treesum: Implement --accept-treesum: trust a treesum-file for a directory checksum Franz Glasner <fzglas.hg@dom66.de> parents: 307 diff changeset	2032
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	2033 if __name__ == "__main__":
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	2034 sys.exit(main())

Mercurial > hgrepos > Python > apps > py-cutils

annotate cutils/treesum.py @ 341:728ad9c639f2