Mercurial > hgrepos > Python > apps > py-cutils
annotate cutils/util/constants.py @ 177:089c40240061
Add an alternate implementation for generating directory tree digests:
- Do not use something like os.walk() but use os.scandir() directly.
- Recursively generate the subdirectory digests only when needed and in
the right order.
This fixes that the order of subdirectories in the output did not
match the application order of its directory digests.
The new implementation also should make filtering (that will be
implemented later) easier.
NOTE: The tree digests of the old and the new implementation are identical.
| author | Franz Glasner <fzglas.hg@dom66.de> |
|---|---|
| date | Sat, 11 Jan 2025 17:41:28 +0100 |
| parents | a813094ae4f5 |
| children | 0f4febf646f5 |
| rev | line source |
|---|---|
|
117
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
1 # -*- coding: utf-8 -*- |
|
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
2 # :- |
|
126
dfe7bb0579e2
Extend copyright years to 2025
Franz Glasner <fzglas.hg@dom66.de>
parents:
117
diff
changeset
|
3 # :Copyright: (c) 2020-2025 Franz Glasner |
|
117
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
4 # :License: BSD-3-Clause |
|
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
5 # :- |
|
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
6 r"""Common constants and compatibility definitions. |
|
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
7 |
|
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
8 """ |
|
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
9 |
|
164
a813094ae4f5
Move PY2 from cutils.util.constants into cutils.util
Franz Glasner <fzglas.hg@dom66.de>
parents:
126
diff
changeset
|
10 __all__ = ["PATH_TYPES", |
|
117
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
11 "READ_CHUNK_SIZE", |
|
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
12 "MAX_AUTO_MAP_SIZE", |
|
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
13 "MAP_WINDOW_SIZE" |
|
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
14 ] |
|
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
15 |
|
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
16 |
|
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
17 try: |
|
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
18 import pathlib |
|
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
19 except ImportError: |
|
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
20 pathlib = None |
|
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
21 |
|
164
a813094ae4f5
Move PY2 from cutils.util.constants into cutils.util
Franz Glasner <fzglas.hg@dom66.de>
parents:
126
diff
changeset
|
22 from . import PY2 |
|
117
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
23 |
|
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
24 |
|
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
25 if PY2: |
|
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
26 PATH_TYPES = (unicode, str) # noqa: F821 (undefined name 'unicode') |
|
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
27 else: |
|
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
28 if pathlib: |
|
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
29 PATH_TYPES = (str, bytes, pathlib.Path) |
|
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
30 else: |
|
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
31 PATH_TYPES = (str, bytes) |
|
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
32 |
|
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
33 READ_CHUNK_SIZE = 2 * 1024 * 1024 # like BUFSIZE_MAX on FreeBSD |
|
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
34 MAX_AUTO_MAP_SIZE = 8 * 1024 * 1024 |
|
e51f34ad6d71
Move constant definitions into new module cutils.util.constants
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff
changeset
|
35 MAP_WINDOW_SIZE = MAX_AUTO_MAP_SIZE # do not totally trash memory on big files |
