Python/apps/py-cutils: cutils/treesum.py annotate

annotate cutils/treesum.py @ 151:b26c4290e928

Implement "--mtime" for treesum to include a file's mtime in a directory digest. Pure file diests are not affected.

author	Franz Glasner <fzglas.hg@dom66.de>
date	Fri, 03 Jan 2025 23:33:37 +0100
parents	f84cf853da22
children	46cb438fa520

rev	line source
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	1 # -- coding: utf-8 --
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	2 # :-
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	3 # :Copyright: (c) 2020-2025 Franz Glasner
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	4 # :License: BSD-3-Clause
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	5 # :-
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	6 r"""Generate and verify checksums for directory trees.
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	7
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	8 """
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	9
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	10 from __future__ import print_function, absolute_import
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	11
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	12
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	13 __all__ = []
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	14
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	15
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	16 import argparse
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	17 import base64
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	18 import binascii
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	19 import datetime
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	20 import os
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	21 import sys
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	22 import time
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	23
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	24 from . import (__version__, __revision__)
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	25 from . import util
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	26 from .util import cm
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	27 from .util import digest
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	28 from .util import walk
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	29
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	30
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	31 def main(argv=None):
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	32
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	33 def _populate_generate_arguments(gp):
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	34 """Use to populate command aliases.
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	35
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	36 This is because :class:`argparse.ArgumentParser` does not
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	37 support them for all supported Python versions.
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	38
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	39 """
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	40 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	41 "--algorithm", "-a", action="store", type=util.argv2algo,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	42 help="1 (aka sha1), 224, 256, 384, 512, "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	43 "3 (alias for sha3-512), 3-224, 3-256, 3-384, 3-512, "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	44 "blake2b, blake2b-256 (default), blake2s, "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	45 "blake2 (alias for blake2b), "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	46 "blake2-256 (alias for blake2b-256), "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	47 "md5")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	48 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	49 "--append-output", action="store_true", dest="append_output",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	50 help="Append to the output file instead of overwriting it.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	51 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	52 "--base64", action="store_true",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	53 help="Output checksums in base64 notation, not hexadecimal "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	54 "(OpenBSD).")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	55 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	56 "--comment", action="append", default=[],
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	57 help="Put given comment COMMENT into the output as \"COMMENT\". "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	58 "Can be given more than once.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	59 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	60 "--follow-directory-symlinks", "-l", action="store_true",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	61 dest="follow_directory_symlinks",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	62 help="Follow symbolic links to directories when walking a "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	63 "directory tree. Note that this is different from using "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	64 "\"--logical\" or \"--physical\" for arguments given "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	65 "directly on the command line")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	66 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	67 "--logical", "-L", dest="logical", action="store_true",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	68 default=None,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	69 help="Follow symbolic links given on command line arguments."
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	70 " Note that this is a different setting as to follow symbolic"
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	71 " links to directories when traversing a directory tree.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	72 gp.add_argument(
150 f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	73 "--minimal", nargs="?", const="", default=None, metavar="TAG",
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	74 help="Produce minimal output only. If a TAG is given and not "
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	75 "empty use it as the leading \"ROOT (<TAG>)\" output.")
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	76 gp.add_argument(
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	77 "--mmap", action="store_true", dest="mmap", default=None,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	78 help="Use mmap if available. Default is to determine "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	79 "automatically from the filesize.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	80 gp.add_argument(
151 b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	81 "--mtime", action="store_true", dest="metadata_mtime",
b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	82 help="Consider the mtime of files (non-directories) when "
b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	83 "generating digests for directories. Digests for files are "
b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	84 "not affected.")
b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	85 gp.add_argument(
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	86 "--no-mmap", action="store_false", dest="mmap", default=None,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	87 help="Dont use mmap.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	88 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	89 "--output", "-o", action="store", metavar="OUTPUT",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	90 help="Put the checksum into given file. "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	91 "If not given or if it is given as `-' then stdout is used.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	92 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	93 "--physical", "-P", dest="logical", action="store_false",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	94 default=None,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	95 help="Do not follow symbolic links given on comment line "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	96 "arguments. This is the default.")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	97 gp.add_argument(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	98 "directories", nargs="*", metavar="DIRECTORY")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	99
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	100 parser = argparse.ArgumentParser(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	101 description="Generate and verify checksums for directory trees.",
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	102 fromfile_prefix_chars='@',
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	103 add_help=False)
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	104
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	105 #
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	106 # Global options for all sub-commands
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	107 #
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	108 gparser = parser.add_argument_group(title="Global Options")
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	109 gparser.add_argument(
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	110 "-v", "--version", action="version",
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	111 version="%s (rv:%s)" % (__version__, __revision__),
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	112 help="Show program's version number and exit")
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	113 gparser.add_argument(
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	114 "-h", "--help", action="help",
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	115 help="Show this help message and exit")
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	116
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	117 #
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	118 # Subcommands
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	119 #
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	120 subparsers = parser.add_subparsers(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	121 dest="subcommand",
145 073e0faea599 Optimize help output for subcommands Franz Glasner <fzglas.hg@dom66.de> parents: 144 diff changeset	122 title="Commands",
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	123 description="This tool uses subcommands. "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	124 "To see detailed help for a specific subcommand use "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	125 "the -h/--help option after the subcommand name. "
148 17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	126 "A list of valid commands and their short descriptions "
17d6e760143f Optimize help / usage output for the global options. Franz Glasner <fzglas.hg@dom66.de> parents: 147 diff changeset	127 "is listed below:",
145 073e0faea599 Optimize help output for subcommands Franz Glasner <fzglas.hg@dom66.de> parents: 144 diff changeset	128 metavar="COMMAND")
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	129
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	130 genparser = subparsers.add_parser(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	131 "generate",
143 492975912cad Wording Franz Glasner <fzglas.hg@dom66.de> parents: 142 diff changeset	132 help="Generate checksums for directory trees.",
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	133 description="Generate checksums for directory trees")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	134 _populate_generate_arguments(genparser)
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	135 # And an alias for "generate"
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	136 genparser2 = subparsers.add_parser(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	137 "gen",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	138 help="Alias for \"generate\"",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	139 description="Generate checksums for directory trees. "
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	140 "This is an alias to \"generate\".")
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	141 _populate_generate_arguments(genparser2)
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	142
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	143 hparser = subparsers.add_parser(
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	144 "help",
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	145 help="Show this help message or a subcommand's help and exit",
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	146 description="Show this help message or a subcommand's help and exit.")
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	147 hparser.add_argument("help_command", nargs='?', metavar="COMMAND")
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	148
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	149 vparser = subparsers.add_parser(
144 b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	150 "version",
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	151 help="Show the program's version number and exit",
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	152 description="Show the program's version number and exit.")
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	153
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	154 # Parse leniently to just check for "version" and/or help
146 7d8df8311e3b Optimize argument parsing for the "version" command Franz Glasner <fzglas.hg@dom66.de> parents: 145 diff changeset	155 opts, _dummy = parser.parse_known_args(args=argv)
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	156
144 b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	157 if opts.subcommand == "version":
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	158 print("%s (rv:%s)" % (__version__, __revision__),
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	159 file=sys.stdout)
b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	160 sys.exit(0)
147 ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	161 if opts.subcommand == "help":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	162 if not opts.help_command:
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	163 parser.print_help()
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	164 else:
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	165 if opts.help_command == "generate":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	166 genparser.print_help()
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	167 elif opts.help_command == "gen":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	168 genparser2.print_help()
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	169 elif opts.help_command == "version":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	170 vparser.print_help()
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	171 elif opts.help_command == "help":
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	172 hparser.print_help()
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	173 else:
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	174 parser.print_help()
ed35f3c9e2b5 Add also a "help" subcommand to "treesum". Franz Glasner <fzglas.hg@dom66.de> parents: 146 diff changeset	175 sys.exit(0)
144 b39f8082ced1 Make a "version" subcommand to also print the program's version number for "treesum" Franz Glasner <fzglas.hg@dom66.de> parents: 143 diff changeset	176
146 7d8df8311e3b Optimize argument parsing for the "version" command Franz Glasner <fzglas.hg@dom66.de> parents: 145 diff changeset	177 # Reparse strictly
7d8df8311e3b Optimize argument parsing for the "version" command Franz Glasner <fzglas.hg@dom66.de> parents: 145 diff changeset	178 opts = parser.parse_args(args=argv)
7d8df8311e3b Optimize argument parsing for the "version" command Franz Glasner <fzglas.hg@dom66.de> parents: 145 diff changeset	179
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	180 return treesum(opts)
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	181
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	182
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	183 def gen_generate_opts(directories=[],
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	184 algorithm="BLAKE2b-256",
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	185 append_output=False,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	186 base64=False,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	187 comment=[],
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	188 follow_directory_symlinks=False,
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	189 logical=None,
150 f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	190 minimal=None,
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	191 mmap=None,
151 b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	192 mtime=False,
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	193 output=None):
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	194 opts = argparse.Namespace(
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	195 directories=directories,
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	196 algorithm=(util.algotag2algotype(algorithm),
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	197 algorithm),
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	198 append_output=append_output,
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	199 base64=base64,
135 dbf27681a1f6 Allow to put comments into the output with "--comment" Franz Glasner <fzglas.hg@dom66.de> parents: 134 diff changeset	200 comment=comment,
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	201 follow_directory_symlinks=follow_directory_symlinks,
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	202 logical=logical,
150 f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	203 minimal=minimal,
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	204 mmap=mmap,
151 b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	205 metadata_mtime=mtime,
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	206 output=output)
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	207 return opts
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	208
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	209
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	210 def treesum(opts):
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	211 # XXX TBD: opts.check and opts.checklist (as in shasum.py)
142 71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	212 if opts.subcommand in ("generate", "gen"):
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	213 return generate_treesum(opts)
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	214 else:
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	215 raise RuntimeError(
71747e45b52c Prepare for using subcommands in "treesum.py". Franz Glasner <fzglas.hg@dom66.de> parents: 137 diff changeset	216 "command `{}' not yet handled".format(opts.subcommand))
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	217
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	218
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	219 def generate_treesum(opts):
149 f717854be1de Put the defaults generation when generating directory digests into "generate_treesum()" instead of "main()" Franz Glasner <fzglas.hg@dom66.de> parents: 148 diff changeset	220 # Provide defaults
f717854be1de Put the defaults generation when generating directory digests into "generate_treesum()" instead of "main()" Franz Glasner <fzglas.hg@dom66.de> parents: 148 diff changeset	221 if not opts.algorithm:
f717854be1de Put the defaults generation when generating directory digests into "generate_treesum()" instead of "main()" Franz Glasner <fzglas.hg@dom66.de> parents: 148 diff changeset	222 opts.algorithm = util.argv2algo("blake2b-256")
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	223 if not opts.directories:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	224 opts.directories.append(".")
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	225
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	226 if opts.output is None or opts.output == "-":
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	227 if hasattr(sys.stdout, "buffer"):
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	228 out_cm = cm.nullcontext(sys.stdout.buffer)
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	229 else:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	230 out_cm = cm.nullcontext(sys.stdout)
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	231 else:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	232 if opts.append_output:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	233 out_cm = open(opts.output, "ab")
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	234 else:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	235 out_cm = open(opts.output, "wb")
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	236
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	237 with out_cm as outfp:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	238 for d in opts.directories:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	239 generate_treesum_for_directory(
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	240 outfp, d, opts.algorithm, opts.mmap, opts.base64, opts.logical,
135 dbf27681a1f6 Allow to put comments into the output with "--comment" Franz Glasner <fzglas.hg@dom66.de> parents: 134 diff changeset	241 opts.follow_directory_symlinks,
151 b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	242 opts.metadata_mtime,
150 f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	243 minimal=opts.minimal,
135 dbf27681a1f6 Allow to put comments into the output with "--comment" Franz Glasner <fzglas.hg@dom66.de> parents: 134 diff changeset	244 comment=opts.comment)
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	245
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	246
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	247 def generate_treesum_for_directory(
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	248 outfp, root, algorithm, use_mmap, use_base64, handle_root_logical,
151 b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	249 follow_directory_symlinks, with_metadata_mtime,
b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	250 minimal=None, comment=None):
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	251 """
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	252
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	253 :param outfp: a binary file with a "write()" and a "flush()" method
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	254
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	255 """
134 d2c303695fb8 Put a file version number into the output. Franz Glasner <fzglas.hg@dom66.de> parents: 133 diff changeset	256 outfp.write(format_bsd_line("VERSION", "1", None, False))
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	257 outfp.flush()
125 12d6ec1f8613 Implement "--logical" and "--physical" to control following symlinked directories when given on the commandline Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	258
12d6ec1f8613 Implement "--logical" and "--physical" to control following symlinked directories when given on the commandline Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	259 # Note given non-default flags that are relevant for directory traversal
12d6ec1f8613 Implement "--logical" and "--physical" to control following symlinked directories when given on the commandline Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	260 flags = []
151 b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	261 if with_metadata_mtime:
b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	262 flags.append("with-metadata-mtime")
125 12d6ec1f8613 Implement "--logical" and "--physical" to control following symlinked directories when given on the commandline Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	263 if handle_root_logical:
129 bdd8ea43074b Output FLAGS as line "FLAGS = ..." instead of "FLAGS (...)" Franz Glasner <fzglas.hg@dom66.de> parents: 128 diff changeset	264 flags.append("logical")
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	265 if follow_directory_symlinks:
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	266 flags.append("follow-directory-symlinks")
125 12d6ec1f8613 Implement "--logical" and "--physical" to control following symlinked directories when given on the commandline Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	267 if flags:
129 bdd8ea43074b Output FLAGS as line "FLAGS = ..." instead of "FLAGS (...)" Franz Glasner <fzglas.hg@dom66.de> parents: 128 diff changeset	268 outfp.write(format_bsd_line("FLAGS", ",".join(flags), None, False))
134 d2c303695fb8 Put a file version number into the output. Franz Glasner <fzglas.hg@dom66.de> parents: 133 diff changeset	269 outfp.flush()
125 12d6ec1f8613 Implement "--logical" and "--physical" to control following symlinked directories when given on the commandline Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	270
150 f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	271 if minimal is None:
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	272 # Write execution timestamps in POSIX epoch and ISO format
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	273 ts = time.time()
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	274 outfp.write(format_bsd_line("TIMESTAMP", ts, None, False))
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	275 ts = (datetime.datetime.utcfromtimestamp(ts)).isoformat("T")
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	276 outfp.write(format_bsd_line("ISOTIMESTAMP", ts, None, False))
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	277 outfp.flush()
134 d2c303695fb8 Put a file version number into the output. Franz Glasner <fzglas.hg@dom66.de> parents: 133 diff changeset	278
150 f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	279 if comment:
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	280 for line in comment:
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	281 outfp.write(format_bsd_line("COMMENT", None, line, False))
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	282
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	283 if minimal is not None:
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	284 outfp.write(
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	285 format_bsd_line(
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	286 "ROOT", None, minimal if minimal else "", False))
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	287 else:
f84cf853da22 Implement "--minimal [TAG]" for treesum. Franz Glasner <fzglas.hg@dom66.de> parents: 149 diff changeset	288 outfp.write(format_bsd_line("ROOT", None, root, False))
134 d2c303695fb8 Put a file version number into the output. Franz Glasner <fzglas.hg@dom66.de> parents: 133 diff changeset	289 outfp.flush()
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	290
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	291 dir_digests = {}
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	292
125 12d6ec1f8613 Implement "--logical" and "--physical" to control following symlinked directories when given on the commandline Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	293 if not handle_root_logical and os.path.islink(root):
12d6ec1f8613 Implement "--logical" and "--physical" to control following symlinked directories when given on the commandline Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	294 linktgt = util.fsencode(os.readlink(root))
12d6ec1f8613 Implement "--logical" and "--physical" to control following symlinked directories when given on the commandline Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	295 linkdgst = algorithm[0]()
132 8b73dca5db97 Encode the link targets as netstrings also before digesting them Franz Glasner <fzglas.hg@dom66.de> parents: 131 diff changeset	296 linkdgst.update(b"%d:%s," % (len(linktgt), linktgt))
125 12d6ec1f8613 Implement "--logical" and "--physical" to control following symlinked directories when given on the commandline Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	297 dir_dgst = algorithm[0]()
137 69f8a3db8fbd Change the tag for symlinks given on the command line: no "filename" here Franz Glasner <fzglas.hg@dom66.de> parents: 136 diff changeset	298 dir_dgst.update(b"1:L,")
125 12d6ec1f8613 Implement "--logical" and "--physical" to control following symlinked directories when given on the commandline Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	299 dir_dgst.update(linkdgst.digest())
133 cf4fb9a33f94 Style Franz Glasner <fzglas.hg@dom66.de> parents: 132 diff changeset	300 outfp.write(
cf4fb9a33f94 Style Franz Glasner <fzglas.hg@dom66.de> parents: 132 diff changeset	301 format_bsd_line(
cf4fb9a33f94 Style Franz Glasner <fzglas.hg@dom66.de> parents: 132 diff changeset	302 algorithm[1],
cf4fb9a33f94 Style Franz Glasner <fzglas.hg@dom66.de> parents: 132 diff changeset	303 dir_dgst.digest(),
cf4fb9a33f94 Style Franz Glasner <fzglas.hg@dom66.de> parents: 132 diff changeset	304 "./@",
cf4fb9a33f94 Style Franz Glasner <fzglas.hg@dom66.de> parents: 132 diff changeset	305 use_base64))
125 12d6ec1f8613 Implement "--logical" and "--physical" to control following symlinked directories when given on the commandline Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	306 outfp.flush()
12d6ec1f8613 Implement "--logical" and "--physical" to control following symlinked directories when given on the commandline Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	307 return
12d6ec1f8613 Implement "--logical" and "--physical" to control following symlinked directories when given on the commandline Franz Glasner <fzglas.hg@dom66.de> parents: 124 diff changeset	308
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	309 for top, dirs, nondirs in walk.walk(
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	310 root,
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	311 follow_symlinks=follow_directory_symlinks):
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	312 dir_dgst = algorithm[0]()
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	313 for dn in dirs:
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	314 if dn.is_symlink and not follow_directory_symlinks:
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	315 linktgt = util.fsencode(os.readlink(dn.path))
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	316 linkdgst = algorithm[0]()
132 8b73dca5db97 Encode the link targets as netstrings also before digesting them Franz Glasner <fzglas.hg@dom66.de> parents: 131 diff changeset	317 linkdgst.update(b"%d:%s," % (len(linktgt), linktgt))
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	318 dir_dgst.update(b"1:S,%d:%s," % (len(dn.fsname), dn.fsname))
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	319 dir_dgst.update(linkdgst.digest())
127 6a50d02fe0ca Change the filename output: make it more consistent Franz Glasner <fzglas.hg@dom66.de> parents: 125 diff changeset	320 opath = "/".join(top) + "/" + dn.name if top else dn.name
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	321 outfp.write(
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	322 format_bsd_line(
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	323 algorithm[1],
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	324 linkdgst.digest(),
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	325 "%s/./@" % (opath,),
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	326 use_base64))
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	327 outfp.flush()
131 3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	328 continue
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	329 # fetch from dir_digests
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	330 dgst = dir_digests[top + (dn.name,)]
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	331 dir_dgst.update(b"1:d,%d:%s," % (len(dn.fsname), dn.fsname))
3a18d71d7c50 Implement --follow-directory-symlinks when walking a directory tree Franz Glasner <fzglas.hg@dom66.de> parents: 130 diff changeset	332 dir_dgst.update(dgst)
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	333 for fn in nondirs:
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	334 dir_dgst.update(b"1:f,%d:%s," % (len(fn.fsname), fn.fsname))
151 b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	335 if with_metadata_mtime:
b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	336 mtime = datetime.datetime.utcfromtimestamp(
b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	337 int(fn.stat.st_mtime))
b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	338 mtime = mtime.isoformat("T") + "Z"
b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	339 if not isinstance(mtime, bytes):
b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	340 mtime = mtime.encode("ascii")
b26c4290e928 Implement "--mtime" for treesum to include a file's mtime in a directory digest. Franz Glasner <fzglas.hg@dom66.de> parents: 150 diff changeset	341 dir_dgst.update(b"5:mtime,%d:%s," % (len(mtime), mtime))
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	342 dgst = digest.compute_digest_file(
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	343 algorithm[0], fn.path, use_mmap=use_mmap)
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	344 dir_dgst.update(dgst)
127 6a50d02fe0ca Change the filename output: make it more consistent Franz Glasner <fzglas.hg@dom66.de> parents: 125 diff changeset	345 opath = "/".join(top) + "/" + fn.name if top else fn.name
133 cf4fb9a33f94 Style Franz Glasner <fzglas.hg@dom66.de> parents: 132 diff changeset	346 outfp.write(
cf4fb9a33f94 Style Franz Glasner <fzglas.hg@dom66.de> parents: 132 diff changeset	347 format_bsd_line(
cf4fb9a33f94 Style Franz Glasner <fzglas.hg@dom66.de> parents: 132 diff changeset	348 algorithm[1], dgst, opath, use_base64))
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	349 outfp.flush()
127 6a50d02fe0ca Change the filename output: make it more consistent Franz Glasner <fzglas.hg@dom66.de> parents: 125 diff changeset	350 opath = "/".join(top) + "/" if top else ""
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	351 outfp.write(format_bsd_line(
127 6a50d02fe0ca Change the filename output: make it more consistent Franz Glasner <fzglas.hg@dom66.de> parents: 125 diff changeset	352 algorithm[1], dir_dgst.digest(), opath, use_base64))
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	353 outfp.flush()
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	354 dir_digests[top] = dir_dgst.digest()
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	355
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	356
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	357 def format_bsd_line(digestname, value, filename, use_base64):
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	358 ls = os.linesep if isinstance(os.linesep, bytes) \
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	359 else os.linesep.encode("utf-8")
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	360 if not isinstance(digestname, bytes):
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	361 digestname = digestname.encode("ascii")
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	362 if digestname == b"TIMESTAMP":
129 bdd8ea43074b Output FLAGS as line "FLAGS = ..." instead of "FLAGS (...)" Franz Glasner <fzglas.hg@dom66.de> parents: 128 diff changeset	363 assert filename is None
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	364 return b"TIMESTAMP = %d%s" % (value, ls)
134 d2c303695fb8 Put a file version number into the output. Franz Glasner <fzglas.hg@dom66.de> parents: 133 diff changeset	365 if digestname in (b"ISOTIMESTAMP", b"FLAGS", b"VERSION"):
129 bdd8ea43074b Output FLAGS as line "FLAGS = ..." instead of "FLAGS (...)" Franz Glasner <fzglas.hg@dom66.de> parents: 128 diff changeset	366 assert filename is None
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	367 if not isinstance(value, bytes):
7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	368 value = value.encode("ascii")
129 bdd8ea43074b Output FLAGS as line "FLAGS = ..." instead of "FLAGS (...)" Franz Glasner <fzglas.hg@dom66.de> parents: 128 diff changeset	369 return b"%s = %s%s" % (digestname, value, ls)
bdd8ea43074b Output FLAGS as line "FLAGS = ..." instead of "FLAGS (...)" Franz Glasner <fzglas.hg@dom66.de> parents: 128 diff changeset	370 assert filename is not None
135 dbf27681a1f6 Allow to put comments into the output with "--comment" Franz Glasner <fzglas.hg@dom66.de> parents: 134 diff changeset	371 if digestname == b"COMMENT":
dbf27681a1f6 Allow to put comments into the output with "--comment" Franz Glasner <fzglas.hg@dom66.de> parents: 134 diff changeset	372 if not isinstance(filename, bytes):
dbf27681a1f6 Allow to put comments into the output with "--comment" Franz Glasner <fzglas.hg@dom66.de> parents: 134 diff changeset	373 filename = filename.encode("utf-8")
dbf27681a1f6 Allow to put comments into the output with "--comment" Franz Glasner <fzglas.hg@dom66.de> parents: 134 diff changeset	374 return b"COMMENT (%s)%s" % (filename, ls)
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	375 if not isinstance(filename, bytes):
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	376 filename = util.fsencode(filename)
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	377 if value is None:
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	378 return b"%s (%s)%s" % (digestname, filename, ls)
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	379 if use_base64:
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	380 value = base64.b64encode(value)
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	381 else:
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	382 value = binascii.hexlify(value)
130 d5621028ce39 Change again the filename tags to be used for symlinks Franz Glasner <fzglas.hg@dom66.de> parents: 129 diff changeset	383 if filename != b"./@":
127 6a50d02fe0ca Change the filename output: make it more consistent Franz Glasner <fzglas.hg@dom66.de> parents: 125 diff changeset	384 filename = util.normalize_filename(filename, True)
128 7c646921a479 Add TIMESTAMP and ISOTIMESTAMP to the output Franz Glasner <fzglas.hg@dom66.de> parents: 127 diff changeset	385 return b"%s (%s) = %s%s" % (digestname, filename, value, ls)
124 3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	386
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	387
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	388 if __name__ == "__main__":
3bd3f32b5e60 A first version of "treesum" is working Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	389 sys.exit(main())

Mercurial > hgrepos > Python > apps > py-cutils

annotate cutils/treesum.py @ 151:b26c4290e928