Python/apps/py-cutils: cutils/treesum.py annotate

annotate cutils/treesum.py @ 147:ed35f3c9e2b5

Add also a "help" subcommand to "treesum". It can show the global general help or the help message for a given subcommand.

author	Franz Glasner <fzglas.hg@dom66.de>
date	Fri, 03 Jan 2025 16:51:26 +0100
parents	7d8df8311e3b
children	17d6e760143f

rev	line source
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	1 # -- coding: utf-8 --
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	2 # :-
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	3 # :Copyright: (c) 2020-2025 Franz Glasner
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	4 # :License: BSD-3-Clause
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	5 # :-
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	6 r"""Generate and verify checksums for directory trees.
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	7
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	8 """
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	9
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	10 from __future__ import print_function, absolute_import
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	11
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	12
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	13 __all__ = []
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	14
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	15
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	16 import argparse
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	17 import base64
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	18 import binascii
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	19 import datetime
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	20 import os
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	21 import sys
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	22 import time
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	23
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	24 from . import (__version__, __revision__)
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	25 from . import util
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	26 from .util import cm
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	27 from .util import digest
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	28 from .util import walk
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	29
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	30
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	31 def main(argv=None):
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	32
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	33 def _populate_generate_arguments(gp):
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	34 """Use to populate command aliases.
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	35
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	36 This is because :class:`argparse.ArgumentParser` does not
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	37 support them for all supported Python versions.
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	38
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	39 """
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	40 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	41 "--algorithm", "-a", action="store", type=util.argv2algo,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	42 help="1 (aka sha1), 224, 256, 384, 512, "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	43 "3 (alias for sha3-512), 3-224, 3-256, 3-384, 3-512, "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	44 "blake2b, blake2b-256 (default), blake2s, "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	45 "blake2 (alias for blake2b), "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	46 "blake2-256 (alias for blake2b-256), "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	47 "md5")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	48 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	49 "--append-output", action="store_true", dest="append_output",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	50 help="Append to the output file instead of overwriting it.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	51 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	52 "--base64", action="store_true",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	53 help="Output checksums in base64 notation, not hexadecimal "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	54 "(OpenBSD).")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	55 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	56 "--comment", action="append", default=[],
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	57 help="Put given comment COMMENT into the output as \"COMMENT\". "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	58 "Can be given more than once.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	59 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	60 "--follow-directory-symlinks", "-l", action="store_true",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	61 dest="follow_directory_symlinks",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	62 help="Follow symbolic links to directories when walking a "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	63 "directory tree. Note that this is different from using "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	64 "\"--logical\" or \"--physical\" for arguments given "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	65 "directly on the command line")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	66 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	67 "--logical", "-L", dest="logical", action="store_true",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	68 default=None,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	69 help="Follow symbolic links given on command line arguments."
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	70 " Note that this is a different setting as to follow symbolic"
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	71 " links to directories when traversing a directory tree.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	72 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	73 "--mmap", action="store_true", dest="mmap", default=None,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	74 help="Use mmap if available. Default is to determine "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	75 "automatically from the filesize.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	76 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	77 "--no-mmap", action="store_false", dest="mmap", default=None,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	78 help="Dont use mmap.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	79 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	80 "--output", "-o", action="store", metavar="OUTPUT",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	81 help="Put the checksum into given file. "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	82 "If not given or if it is given as `-' then stdout is used.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	83 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	84 "--physical", "-P", dest="logical", action="store_false",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	85 default=None,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	86 help="Do not follow symbolic links given on comment line "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	87 "arguments. This is the default.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	88 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	89 "directories", nargs="*", metavar="DIRECTORY")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	90
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	91 parser = argparse.ArgumentParser(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	92 description="Generate and verify checksums for directory trees.",
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	93 fromfile_prefix_chars='@')
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	94
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	95 # Global options for all sub-commands
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	96 parser.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	97 "-v", "--version", action="version",
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	98 version="%s (rv:%s)" % (__version__, __revision__))
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	99
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	100 subparsers = parser.add_subparsers(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	101 dest="subcommand",
145 073e0faea599 Optimize help output for subcommands Franz Glasner <fzglas.hg@dom66.de> parents: 144 diff changeset	102 title="Commands",
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	103 description="This tool uses subcommands. "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	104 "To see detailed help for a specific subcommand use "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	105 "the -h/--help option after the subcommand name. "
145 073e0faea599 Optimize help output for subcommands Franz Glasner <fzglas.hg@dom66.de> parents: 144 diff changeset	106 "A short list of valid commands are listed below:",
073e0faea599 Optimize help output for subcommands Franz Glasner <fzglas.hg@dom66.de> parents: 144 diff changeset	107 metavar="COMMAND")
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	108
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	109 genparser = subparsers.add_parser(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	110 "generate",
143 492975912cad Wording Franz Glasner <fzglas.hg@dom66.de> parents: 142 diff changeset	111 help="Generate checksums for directory trees.",
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	112 description="Generate checksums for directory trees")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	113 _populate_generate_arguments(genparser)
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	114 # And an alias for "generate"
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	115 genparser2 = subparsers.add_parser(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	116 "gen",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	117 help="Alias for \"generate\"",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	118 description="Generate checksums for directory trees. "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	119 "This is an alias to \"generate\".")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	120 _populate_generate_arguments(genparser2)
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	121
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	122 hparser = subparsers.add_parser(
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	123 "help",
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	124 help="Show this help message or a subcommand's help and exit",
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	125 description="Show this help message or a subcommand's help and exit.")
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	126 hparser.add_argument("help_command", nargs='?', metavar="COMMAND")
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	127
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	128 vparser = subparsers.add_parser(
144 b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	129 "version",
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	130 help="Show the program's version number and exit",
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	131 description="Show the program's version number and exit.")
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	132
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	133 # Parse leniently to just check for "version" and/or help
146 7d8df8311e3b Optimize argument parsing for the "version" command Franz Glasner <fzglas.hg@dom66.de> parents: 145 diff changeset	134 opts, _dummy = parser.parse_known_args(args=argv)
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	135
144 b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	136 if opts.subcommand == "version":
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	137 print("%s (rv:%s)" % (__version__, __revision__),
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	138 file=sys.stdout)
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	139 sys.exit(0)
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	140 if opts.subcommand == "help":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	141 if not opts.help_command:
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	142 parser.print_help()
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	143 else:
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	144 if opts.help_command == "generate":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	145 genparser.print_help()
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	146 elif opts.help_command == "gen":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	147 genparser2.print_help()
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	148 elif opts.help_command == "version":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	149 vparser.print_help()
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	150 elif opts.help_command == "help":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	151 hparser.print_help()
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	152 else:
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	153 parser.print_help()
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	154 sys.exit(0)
144 b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	155
146 7d8df8311e3b Optimize argument parsing for the "version" command Franz Glasner <fzglas.hg@dom66.de> parents: 145 diff changeset	156 # Reparse strictly
7d8df8311e3b Optimize argument parsing for the "version" command Franz Glasner <fzglas.hg@dom66.de> parents: 145 diff changeset	157 opts = parser.parse_args(args=argv)
7d8df8311e3b Optimize argument parsing for the "version" command Franz Glasner <fzglas.hg@dom66.de> parents: 145 diff changeset	158
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	159 if not opts.algorithm:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	160 opts.algorithm = util.argv2algo("blake2b-256")
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	161
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	162 return treesum(opts)
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	163
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	164
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	165 def gen_generate_opts(directories=[],
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	166 algorithm="BLAKE2b-256",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	167 append_output=False,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	168 base64=False,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	169 comment=[],
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	170 follow_directory_symlinks=False,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	171 logical=None,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	172 mmap=None,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	173 output=None):
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	174 opts = argparse.Namespace(
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	175 directories=directories,
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	176 algorithm=(util.algotag2algotype(algorithm),
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	177 algorithm),
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	178 append_output=append_output,
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	179 base64=base64,
135 dbf27681a1f6 Allow to put comments into the output with "--comment" Franz Glasner <fzglas.hg@dom66.de> parents: 134 diff changeset	180 comment=comment,
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	181 follow_directory_symlinks=follow_directory_symlinks,
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	182 logical=logical,
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	183 mmap=mmap,
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	184 output=output)
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	185 return opts
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	186
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	187
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	188 def treesum(opts):
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	189 # XXX TBD: opts.check and opts.checklist (as in shasum.py)
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	190 if opts.subcommand in ("generate", "gen"):
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	191 return generate_treesum(opts)
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	192 else:
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	193 raise RuntimeError(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	194 "command `{}' not yet handled".format(opts.subcommand))
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	195
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	196
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	197 def generate_treesum(opts):
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	198 if not opts.directories:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	199 opts.directories.append(".")
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	200
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	201 if opts.output is None or opts.output == "-":
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	202 if hasattr(sys.stdout, "buffer"):
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	203 out_cm = cm.nullcontext(sys.stdout.buffer)
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	204 else:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	205 out_cm = cm.nullcontext(sys.stdout)
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	206 else:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	207 if opts.append_output:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	208 out_cm = open(opts.output, "ab")
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	209 else:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	210 out_cm = open(opts.output, "wb")
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	211
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	212 with out_cm as outfp:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	213 for d in opts.directories:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	214 generate_treesum_for_directory(
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	215 outfp, d, opts.algorithm, opts.mmap, opts.base64, opts.logical,
135 dbf27681a1f6 Allow to put comments into the output with "--comment" Franz Glasner <fzglas.hg@dom66.de> parents: 134 diff changeset	216 opts.follow_directory_symlinks,
dbf27681a1f6 Allow to put comments into the output with "--comment" Franz Glasner <fzglas.hg@dom66.de> parents: 134 diff changeset	217 comment=opts.comment)
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	218
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	219
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	220 def generate_treesum_for_directory(
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	221 outfp, root, algorithm, use_mmap, use_base64, handle_root_logical,
135 dbf27681a1f6 Allow to put comments into the output with "--comment" Franz Glasner <fzglas.hg@dom66.de> parents: 134 diff changeset	222 follow_directory_symlinks, comment=None):
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	223 """
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	224
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	225 :param outfp: a binary file with a "write()" and a "flush()" method
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	226
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	227 """
134 d2c303695fb8 Put a file version number into the output. Franz Glasner <fzglas.hg@dom66.de> parents: 133 diff changeset	228 outfp.write(format_bsd_line("VERSION", "1", None, False))
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	229 outfp.flush()
125 12d6ec1f8613 Implement "--logical" and "--physical" to control following symlinked directories when given on the commandline Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	230
12d6ec1f8613 Implement "--logical" and "--physical" to control following symlinked directories when given on the commandline Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	231 # Note given non-default flags that are relevant for directory traversal
12d6ec1f8613 Implement "--logical" and "--physical" to control following symlinked directories when given on the commandline Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	232 flags = []
12d6ec1f8613 Implement "--logical" and "--physical" to control following symlinked directories when given on the commandline Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	233 if handle_root_logical:
129 bdd8ea43074b Output FLAGS as line "FLAGS = ..." instead of "FLAGS (...)" Franz Glasner <fzglas.hg@dom66.de> parents: 128 diff changeset	234 flags.append("logical")
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	235 if follow_directory_symlinks:
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	236 flags.append("follow-directory-symlinks")
125 12d6ec1f8613 Implement "--logical" and "--physical" to control following symlinked directories when given on the commandline Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	237 if flags:
129 bdd8ea43074b Output FLAGS as line "FLAGS = ..." instead of "FLAGS (...)" Franz Glasner <fzglas.hg@dom66.de> parents: 128 diff changeset	238 outfp.write(format_bsd_line("FLAGS", ",".join(flags), None, False))
134 d2c303695fb8 Put a file version number into the output. Franz Glasner <fzglas.hg@dom66.de> parents: 133 diff changeset	239 outfp.flush()
125 12d6ec1f8613 Implement "--logical" and "--physical" to control following symlinked directories when given on the commandline Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	240
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	241 # Write execution timestamps in POSIX epoch and ISO format
7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	242 ts = time.time()
7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	243 outfp.write(format_bsd_line("TIMESTAMP", ts, None, False))
7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	244 ts = (datetime.datetime.utcfromtimestamp(ts)).isoformat("T")
7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	245 outfp.write(format_bsd_line("ISOTIMESTAMP", ts, None, False))
134 d2c303695fb8 Put a file version number into the output. Franz Glasner <fzglas.hg@dom66.de> parents: 133 diff changeset	246 outfp.flush()
d2c303695fb8 Put a file version number into the output. Franz Glasner <fzglas.hg@dom66.de> parents: 133 diff changeset	247
135 dbf27681a1f6 Allow to put comments into the output with "--comment" Franz Glasner <fzglas.hg@dom66.de> parents: 134 diff changeset	248 if comment:
dbf27681a1f6 Allow to put comments into the output with "--comment" Franz Glasner <fzglas.hg@dom66.de> parents: 134 diff changeset	249 for line in comment:
dbf27681a1f6 Allow to put comments into the output with "--comment" Franz Glasner <fzglas.hg@dom66.de> parents: 134 diff changeset	250 outfp.write(format_bsd_line("COMMENT", None, line, False))
134 d2c303695fb8 Put a file version number into the output. Franz Glasner <fzglas.hg@dom66.de> parents: 133 diff changeset	251 outfp.write(format_bsd_line("ROOT", None, root, False))
d2c303695fb8 Put a file version number into the output. Franz Glasner <fzglas.hg@dom66.de> parents: 133 diff changeset	252 outfp.flush()
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	253
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	254 dir_digests = {}
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	255
125 12d6ec1f8613 Implement "--logical" and "--physical" to control following symlinked directories when given on the commandline Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	256 if not handle_root_logical and os.path.islink(root):
12d6ec1f8613 Implement "--logical" and "--physical" to control following symlinked directories when given on the commandline Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	257 linktgt = util.fsencode(os.readlink(root))
12d6ec1f8613 Implement "--logical" and "--physical" to control following symlinked directories when given on the commandline Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	258 linkdgst = algorithm[0]()
132 8b73dca5db97 Encode the link targets as netstrings also before digesting them Franz Glasner <fzglas.hg@dom66.de> parents: 131 diff changeset	259 linkdgst.update(b"%d:%s," % (len(linktgt), linktgt))
125 12d6ec1f8613 Implement "--logical" and "--physical" to control following symlinked directories when given on the commandline Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	260 dir_dgst = algorithm[0]()
137 69f8a3db8fbd Change the tag for symlinks given on the command line: no "filename" here Franz Glasner <fzglas.hg@dom66.de> parents: 136 diff changeset	261 dir_dgst.update(b"1:L,")
125 12d6ec1f8613 Implement "--logical" and "--physical" to control following symlinked directories when given on the commandline Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	262 dir_dgst.update(linkdgst.digest())
133 cf4fb9a33f94 Style Franz Glasner <fzglas.hg@dom66.de> parents: 132 diff changeset	263 outfp.write(
cf4fb9a33f94 Style Franz Glasner <fzglas.hg@dom66.de> parents: 132 diff changeset	264 format_bsd_line(
cf4fb9a33f94 Style Franz Glasner <fzglas.hg@dom66.de> parents: 132 diff changeset	265 algorithm[1],
cf4fb9a33f94 Style Franz Glasner <fzglas.hg@dom66.de> parents: 132 diff changeset	266 dir_dgst.digest(),
cf4fb9a33f94 Style Franz Glasner <fzglas.hg@dom66.de> parents: 132 diff changeset	267 "./@",
cf4fb9a33f94 Style Franz Glasner <fzglas.hg@dom66.de> parents: 132 diff changeset	268 use_base64))
125 12d6ec1f8613 Implement "--logical" and "--physical" to control following symlinked directories when given on the commandline Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	269 outfp.flush()
12d6ec1f8613 Implement "--logical" and "--physical" to control following symlinked directories when given on the commandline Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	270 return
12d6ec1f8613 Implement "--logical" and "--physical" to control following symlinked directories when given on the commandline Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	271
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	272 for top, dirs, nondirs in walk.walk(
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	273 root,
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	274 follow_symlinks=follow_directory_symlinks):
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	275 dir_dgst = algorithm[0]()
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	276 for dn in dirs:
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	277 if dn.is_symlink and not follow_directory_symlinks:
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	278 linktgt = util.fsencode(os.readlink(dn.path))
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	279 linkdgst = algorithm[0]()
132 8b73dca5db97 Encode the link targets as netstrings also before digesting them Franz Glasner <fzglas.hg@dom66.de> parents: 131 diff changeset	280 linkdgst.update(b"%d:%s," % (len(linktgt), linktgt))
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	281 dir_dgst.update(b"1:S,%d:%s," % (len(dn.fsname), dn.fsname))
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	282 dir_dgst.update(linkdgst.digest())
127 6a50d02fe0ca Change the filename output: make it more consistent Franz Glasner <fzglas.hg@dom66.de> parents: 125 diff changeset	283 opath = "/".join(top) + "/" + dn.name if top else dn.name
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	284 outfp.write(
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	285 format_bsd_line(
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	286 algorithm[1],
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	287 linkdgst.digest(),
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	288 "%s/./@" % (opath,),
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	289 use_base64))
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	290 outfp.flush()
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	291 continue
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	292 # fetch from dir_digests
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	293 dgst = dir_digests[top + (dn.name,)]
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	294 dir_dgst.update(b"1:d,%d:%s," % (len(dn.fsname), dn.fsname))
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	295 dir_dgst.update(dgst)
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	296 for fn in nondirs:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	297 dir_dgst.update(b"1:f,%d:%s," % (len(fn.fsname), fn.fsname))
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	298 dgst = digest.compute_digest_file(
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	299 algorithm[0], fn.path, use_mmap=use_mmap)
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	300 dir_dgst.update(dgst)
127 6a50d02fe0ca Change the filename output: make it more consistent Franz Glasner <fzglas.hg@dom66.de> parents: 125 diff changeset	301 opath = "/".join(top) + "/" + fn.name if top else fn.name
133 cf4fb9a33f94 Style Franz Glasner <fzglas.hg@dom66.de> parents: 132 diff changeset	302 outfp.write(
cf4fb9a33f94 Style Franz Glasner <fzglas.hg@dom66.de> parents: 132 diff changeset	303 format_bsd_line(
cf4fb9a33f94 Style Franz Glasner <fzglas.hg@dom66.de> parents: 132 diff changeset	304 algorithm[1], dgst, opath, use_base64))
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	305 outfp.flush()
127 6a50d02fe0ca Change the filename output: make it more consistent Franz Glasner <fzglas.hg@dom66.de> parents: 125 diff changeset	306 opath = "/".join(top) + "/" if top else ""
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	307 outfp.write(format_bsd_line(
127 6a50d02fe0ca Change the filename output: make it more consistent Franz Glasner <fzglas.hg@dom66.de> parents: 125 diff changeset	308 algorithm[1], dir_dgst.digest(), opath, use_base64))
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	309 outfp.flush()
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	310 dir_digests[top] = dir_dgst.digest()
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	311
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	312
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	313 def format_bsd_line(digestname, value, filename, use_base64):
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	314 ls = os.linesep if isinstance(os.linesep, bytes) \
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	315 else os.linesep.encode("utf-8")
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	316 if not isinstance(digestname, bytes):
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	317 digestname = digestname.encode("ascii")
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	318 if digestname == b"TIMESTAMP":
129 bdd8ea43074b Output FLAGS as line "FLAGS = ..." instead of "FLAGS (...)" Franz Glasner <fzglas.hg@dom66.de> parents: 128 diff changeset	319 assert filename is None
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	320 return b"TIMESTAMP = %d%s" % (value, ls)
134 d2c303695fb8 Put a file version number into the output. Franz Glasner <fzglas.hg@dom66.de> parents: 133 diff changeset	321 if digestname in (b"ISOTIMESTAMP", b"FLAGS", b"VERSION"):
129 bdd8ea43074b Output FLAGS as line "FLAGS = ..." instead of "FLAGS (...)" Franz Glasner <fzglas.hg@dom66.de> parents: 128 diff changeset	322 assert filename is None
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	323 if not isinstance(value, bytes):
7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	324 value = value.encode("ascii")
129 bdd8ea43074b Output FLAGS as line "FLAGS = ..." instead of "FLAGS (...)" Franz Glasner <fzglas.hg@dom66.de> parents: 128 diff changeset	325 return b"%s = %s%s" % (digestname, value, ls)
bdd8ea43074b Output FLAGS as line "FLAGS = ..." instead of "FLAGS (...)" Franz Glasner <fzglas.hg@dom66.de> parents: 128 diff changeset	326 assert filename is not None
135 dbf27681a1f6 Allow to put comments into the output with "--comment" Franz Glasner <fzglas.hg@dom66.de> parents: 134 diff changeset	327 if digestname == b"COMMENT":
dbf27681a1f6 Allow to put comments into the output with "--comment" Franz Glasner <fzglas.hg@dom66.de> parents: 134 diff changeset	328 if not isinstance(filename, bytes):
dbf27681a1f6 Allow to put comments into the output with "--comment" Franz Glasner <fzglas.hg@dom66.de> parents: 134 diff changeset	329 filename = filename.encode("utf-8")
dbf27681a1f6 Allow to put comments into the output with "--comment" Franz Glasner <fzglas.hg@dom66.de> parents: 134 diff changeset	330 return b"COMMENT (%s)%s" % (filename, ls)
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	331 if not isinstance(filename, bytes):
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	332 filename = util.fsencode(filename)
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	333 if value is None:
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	334 return b"%s (%s)%s" % (digestname, filename, ls)
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	335 if use_base64:
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	336 value = base64.b64encode(value)
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	337 else:
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	338 value = binascii.hexlify(value)
130 d5621028ce39 Change again the filename tags to be used for symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 129 diff changeset	339 if filename != b"./@":
127 6a50d02fe0ca Change the filename output: make it more consistent Franz Glasner <fzglas.hg@dom66.de> parents: 125 diff changeset	340 filename = util.normalize_filename(filename, True)
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	341 return b"%s (%s) = %s%s" % (digestname, filename, value, ls)
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	342
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	343
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	344 if __name__ == "__main__":
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	345 sys.exit(main())

Mercurial > hgrepos > Python > apps > py-cutils

annotate cutils/treesum.py @ 147:ed35f3c9e2b5