annotate cutils/util/constants.py @ 177:089c40240061

Add an alternate implementation for generating directory tree digests: - Do not use something like os.walk() but use os.scandir() directly. - Recursively generate the subdirectory digests only when needed and in the right order. This fixes that the order of subdirectories in the output did not match the application order of its directory digests. The new implementation also should make filtering (that will be implemented later) easier. NOTE: The tree digests of the old and the new implementation are identical.
author Franz Glasner <fzglas.hg@dom66.de>
date Sat, 11 Jan 2025 17:41:28 +0100
parents a813094ae4f5
children 0f4febf646f5
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
117
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
1 # -*- coding: utf-8 -*-
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
2 # :-
126
dfe7bb0579e2 Extend copyright years to 2025
Franz Glasner <fzglas.hg@dom66.de>
parents: 117
diff changeset
3 # :Copyright: (c) 2020-2025 Franz Glasner
117
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
4 # :License: BSD-3-Clause
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
5 # :-
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
6 r"""Common constants and compatibility definitions.
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
7
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
8 """
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
9
164
a813094ae4f5 Move PY2 from cutils.util.constants into cutils.util
Franz Glasner <fzglas.hg@dom66.de>
parents: 126
diff changeset
10 __all__ = ["PATH_TYPES",
117
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
11 "READ_CHUNK_SIZE",
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
12 "MAX_AUTO_MAP_SIZE",
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
13 "MAP_WINDOW_SIZE"
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
14 ]
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
15
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
16
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
17 try:
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
18 import pathlib
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
19 except ImportError:
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
20 pathlib = None
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
21
164
a813094ae4f5 Move PY2 from cutils.util.constants into cutils.util
Franz Glasner <fzglas.hg@dom66.de>
parents: 126
diff changeset
22 from . import PY2
117
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
23
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
24
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
25 if PY2:
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
26 PATH_TYPES = (unicode, str) # noqa: F821 (undefined name 'unicode')
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
27 else:
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
28 if pathlib:
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
29 PATH_TYPES = (str, bytes, pathlib.Path)
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
30 else:
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
31 PATH_TYPES = (str, bytes)
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
32
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
33 READ_CHUNK_SIZE = 2 * 1024 * 1024 # like BUFSIZE_MAX on FreeBSD
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
34 MAX_AUTO_MAP_SIZE = 8 * 1024 * 1024
e51f34ad6d71 Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
35 MAP_WINDOW_SIZE = MAX_AUTO_MAP_SIZE # do not totally trash memory on big files