Mercurial > hgrepos > Python > apps > py-cutils

annotate cutils/util/walk.py @ 177:089c40240061

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Add an alternate implementation for generating directory tree digests: - Do not use something like os.walk() but use os.scandir() directly. - Recursively generate the subdirectory digests only when needed and in the right order. This fixes that the order of subdirectories in the output did not match the application order of its directory digests. The new implementation also should make filtering (that will be implemented later) easier. NOTE: The tree digests of the old and the new implementation are identical.
author Franz Glasner <fzglas.hg@dom66.de>
date Sat, 11 Jan 2025 17:41:28 +0100
parents 506d895a8500
children dac26a2d9de5
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
121
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
1 # -*- coding: utf-8 -*-
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
2 # :-
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
3 # :Copyright: (c) 2020-2025 Franz Glasner
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
4 # :License: BSD-3-Clause
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
5 # :-
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
6 r"""Utility sub-module to implement a heavily customized :func:`os.walk`.
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
7
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
8 """
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
9
177
089c40240061 Add an alternate implementation for generating directory tree digests:
Franz Glasner <fzglas.hg@dom66.de>
parents: 175
diff changeset
10 __all__ = ["walk",
089c40240061 Add an alternate implementation for generating directory tree digests:
Franz Glasner <fzglas.hg@dom66.de>
parents: 175
diff changeset
11 "ScanDir"]
121
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
12
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
13
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
14 import os
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
15 try:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
16 from os import scandir
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
17 except ImportError:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
18 try:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
19 from scandir import scandir
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
20 except ImportError:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
21 scandir = None
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
22
164
a813094ae4f5 Move PY2 from cutils.util.constants into cutils.util
Franz Glasner <fzglas.hg@dom66.de>
parents: 162
diff changeset
23 from . import PY2
121
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
24 from .cm import nullcontext
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
25
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
26
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
27 class WalkDirEntry(object):
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
28
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
29 """A :class:`os.DirEntry` alike to be used in :func:`walk` and for
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
30 its results.
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
31
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
32 """
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
33
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
34 __slots__ = ("_name", "_fsname", "_path", "_fspath", "_is_symlink",
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
35 "_is_dir", "_stat_result")
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
36
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
37 def __init__(self, name):
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
38 self._name = name
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
39 if PY2:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
40 assert isinstance(name, bytes)
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
41 self._fsname = name
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
42 else:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
43 self._fsname = os.fsencode(name)
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
44 self._path = None
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
45 self._fspath = None
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
46 self._is_symlink = self._is_dir = self._stat_result = None
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
47
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
48 @property
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
49 def name(self):
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
50 return self._name
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
51
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
52 @property
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
53 def fsname(self):
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
54 return self._fsname
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
55
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
56 @property
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
57 def path(self):
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
58 return self._path
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
59
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
60 @property
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
61 def fspath(self):
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
62 if self._path is not None:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
63 if self._fspath is None:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
64 if PY2:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
65 assert isinstance(self._path, bytes)
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
66 self._fspath = self._path
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
67 else:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
68 self._fspath = os.fsencode(self._path)
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
69 return self._fspath
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
70
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
71 @property
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
72 def is_symlink(self):
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
73 return self._is_symlink
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
74
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
75 @property
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
76 def is_dir(self):
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
77 return self._is_dir
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
78
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
79 @property
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
80 def stat(self):
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
81 return self._stat_result
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
82
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
83 def __repr__(self):
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
84 tag = ""
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
85 if self._is_symlink:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
86 tag += "l"
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
87 if self._is_dir:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
88 tag += "d"
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
89 if tag:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
90 return "<WalkDirEntry %r (%s)>" % (self._name, tag)
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
91 return "<WalkDirEntry %r>" % (self._name,)
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
92
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
93 @classmethod
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
94 def from_direntry(cls_, entry):
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
95 w = cls_(entry.name)
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
96 w._path = entry.path
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
97 try:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
98 w._is_dir = entry.is_dir(follow_symlinks=True)
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
99 except OSError:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
100 #
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
101 # If is_dir() raises an OSError, consider that the entry
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
102 # is not a directory, same behaviour than os.path.isdir().
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
103 #
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
104 w._is_dir = False
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
105 try:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
106 w._is_symlink = entry.is_symlink()
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
107 except OSError:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
108 #
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
109 # If is_symlink() raises an OSError, consider that the entry
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
110 # is not a symbolic link, same behaviour than os.path.islink().
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
111 #
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
112 w._is_symlink = False
156
481cc9b26861 Calculate "stat()" for directories also in a WalkDirEntry
Franz Glasner <fzglas.hg@dom66.de>
parents: 155
diff changeset
113 # Do not supress errors here and (consistently) follow symlinks
481cc9b26861 Calculate "stat()" for directories also in a WalkDirEntry
Franz Glasner <fzglas.hg@dom66.de>
parents: 155
diff changeset
114 w._stat_result = entry.stat(follow_symlinks=True)
121
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
115 return w
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
116
162
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
117 @classmethod
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
118 def from_path_name(cls_, path, name):
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
119 w = cls_(name)
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
120 w._path = os.path.join(path, name)
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
121 try:
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
122 w._is_dir = os.path.isdir(w._path)
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
123 except OSError:
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
124 #
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
125 # If is_dir() raises an OSError, consider that the entry
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
126 # is not a directory, same behaviour than os.path.isdir().
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
127 #
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
128 w._is_dir = False
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
129 try:
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
130 w._is_symlink = os.path.islink(w._path)
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
131 except OSError:
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
132 #
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
133 # If is_symlink() raises an OSError, consider that the entry
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
134 # is not a symbolic link, same behaviour than os.path.islink().
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
135 #
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
136 w._is_symlink = False
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
137 w._stat_result = os.stat(w._path)
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
138 return w
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
139
121
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
140 @staticmethod
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
141 def sort_key(entry):
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
142 return entry._fsname
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
143
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
144
162
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
145 def walk(root, follow_symlinks=False):
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
146 """A heyvily customized :func:`os.walk` alike that differs from the
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
147 original:
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
148
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
149 - optimized for use in :command:`treesum`
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
150 - most errors are not suppressed
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
151 - the `root` is never part of the returned data
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
152 - the returned directory in "top" is not a string form but a list of
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
153 individual path segments
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
154 - there is only one yielded list
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
155
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
156 * contains :class:`WalkDirEntry`
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
157 * sorted by its fsname
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
158
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
159 The caller can easily get the old dirs and nondirs by filtering
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
160 the yielded list using "entry.is_dir".
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
161
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
162 - recurse into sub-directories first ("topdown=False")
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
163 - sort consistently all yielded lists by the filesystem encoding
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
164
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
165 .. note:: The implementation is based on Python 3.11 and needs a
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
166 functional :func:`os.scandir` or :func:`scandir.scandir`
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
167 implementation. It intentionally follows the logic in
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
168 Python 3.11 while it could be simplified because we are not
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
169 implementing some of the original flags (e.g. like
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
170 `topdown`).
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
171
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
172 """
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
173 normed_root = os.path.normpath(root)
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
174 yield from _walk(normed_root, tuple(), follow_symlinks=follow_symlinks)
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
175
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
176
121
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
177 if scandir:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
178
162
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
179 def _walk(root, top, follow_symlinks):
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
180 """:func:`walk` helper.
154
c7df81fb84b7 Merge dirs and nondirs into one list that is yielded from "util.walk()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 123
diff changeset
181
162
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
182 Implemented using :func:`os.scandir`.
121
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
183
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
184 """
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
185 if top:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
186 path = os.path.join(root, *top)
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
187 else:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
188 path = root
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
189
154
c7df81fb84b7 Merge dirs and nondirs into one list that is yielded from "util.walk()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 123
diff changeset
190 fsobjects, walk_dirs = [], []
121
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
191
123
4a0c3c9eead7 Use the output of "scandir()" directly if it has a "close" method.
Franz Glasner <fzglas.hg@dom66.de>
parents: 121
diff changeset
192 scandir_cm = scandir(path)
4a0c3c9eead7 Use the output of "scandir()" directly if it has a "close" method.
Franz Glasner <fzglas.hg@dom66.de>
parents: 121
diff changeset
193 if not hasattr(scandir_cm, "close"):
4a0c3c9eead7 Use the output of "scandir()" directly if it has a "close" method.
Franz Glasner <fzglas.hg@dom66.de>
parents: 121
diff changeset
194 scandir_cm = nullcontext(scandir_cm)
4a0c3c9eead7 Use the output of "scandir()" directly if it has a "close" method.
Franz Glasner <fzglas.hg@dom66.de>
parents: 121
diff changeset
195 with scandir_cm as scandir_it:
121
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
196 while True:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
197 try:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
198 entry = WalkDirEntry.from_direntry(next(scandir_it))
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
199 except StopIteration:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
200 break
154
c7df81fb84b7 Merge dirs and nondirs into one list that is yielded from "util.walk()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 123
diff changeset
201 fsobjects.append(entry)
121
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
202 #
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
203 # Always bottom-up: recurse into sub-directories, but exclude
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
204 # symlinks to directories if follow_symlinks is False
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
205 #
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
206 if entry.is_dir:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
207 if follow_symlinks:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
208 walk_into = True
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
209 else:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
210 walk_into = not entry.is_symlink
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
211 if walk_into:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
212 walk_dirs.append(entry)
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
213
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
214 # Sort by low-level filesystem encoding
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
215 walk_dirs.sort(key=WalkDirEntry.sort_key)
154
c7df81fb84b7 Merge dirs and nondirs into one list that is yielded from "util.walk()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 123
diff changeset
216 fsobjects.sort(key=WalkDirEntry.sort_key)
121
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
217
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
218 # Recurse into sub-directories
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
219 for wd in walk_dirs:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
220 yield from _walk(root, top + (wd.name,), follow_symlinks)
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
221 # Yield after recursion if going bottom up
154
c7df81fb84b7 Merge dirs and nondirs into one list that is yielded from "util.walk()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 123
diff changeset
222 yield top, fsobjects
121
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
223
175
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
224
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
225 class ScanDir(object): # noqa: E303 too many blank lines
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
226
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
227 """An :func:`os.scandir` wrapper that is always an iterator and
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
228 a context manager.
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
229
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
230 """
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
231
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
232 __slots__ = ("_scandir_it", )
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
233
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
234 def __init__(self, path):
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
235 super(ScanDir, self).__init__()
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
236 self._scandir_it = os.scandir(path)
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
237
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
238 def __iter__(self):
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
239 return self
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
240
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
241 def __next__(self):
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
242 return WalkDirEntry.from_direntry(next(self._scandir_it))
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
243
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
244 if PY2:
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
245 next = __next__
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
246
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
247 def __enter__(self):
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
248 return self
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
249
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
250 def __exit__(self, *args, **kwds):
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
251 if hasattr(self._scandir_it, "close"):
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
252 self._scandir_it.close()
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
253
121
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
254 else:
2dc26a2f3d1c A heavily customized "os.walk()" alike to support the coming treeview implementation
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
255
162
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
256 def _walk(root, top, follow_symlinks):
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
257 """:func:`walk` helper.
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
258
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
259 Implemented using :func:`os.listdir`.
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
260
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
261 """
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
262 if top:
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
263 path = os.path.join(root, *top)
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
264 else:
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
265 path = root
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
266
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
267 fsobjects, walk_dirs = [], []
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
268
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
269 names = os.listdir(path)
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
270 for name in names:
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
271 entry = WalkDirEntry.from_path_name(path, name)
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
272 fsobjects.append(entry)
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
273 #
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
274 # Always bottom-up: recurse into sub-directories, but exclude
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
275 # symlinks to directories if follow_symlinks is False
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
276 #
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
277 if entry.is_dir:
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
278 if follow_symlinks:
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
279 walk_into = True
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
280 else:
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
281 walk_into = not entry.is_symlink
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
282 if walk_into:
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
283 walk_dirs.append(entry)
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
284
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
285 # Sort by low-level filesystem encoding
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
286 walk_dirs.sort(key=WalkDirEntry.sort_key)
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
287 fsobjects.sort(key=WalkDirEntry.sort_key)
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
288
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
289 # Recurse into sub-directories
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
290 for wd in walk_dirs:
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
291 yield from _walk(root, top + (wd.name,), follow_symlinks)
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
292 # Yield after recursion if going bottom up
29dd5528174c Implement walk._walk() using os.listdir() also.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
293 yield top, fsobjects
175
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
294
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
295
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
296 class ScanDir(object): # noqa: E303 too many blank lines
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
297
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
298 """An :func:`os.scandir` wrapper that is always an iterator and
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
299 a context manager.
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
300
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
301 """
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
302
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
303 __slots__ = ("_listdir_it", "_path")
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
304
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
305 def __init__(self, path):
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
306 super(ScanDir, self).__init__()
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
307 self._listdir_it = iter(os.listdir(path))
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
308 self._path = path
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
309
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
310 def __iter__(self):
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
311 return self
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
312
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
313 def __next__(self):
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
314 return WalkDirEntry.from_path_name(self._path,
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
315 next(self._listdir_it))
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
316
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
317 if PY2:
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
318 next = __next__
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
319
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
320 def __enter__(self):
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
321 return self
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
322
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
323 def __exit__(self, *args, **kwds):
506d895a8500 Implement cutils.util.walk.Scandir as a wrapper for os.scandir()
Franz Glasner <fzglas.hg@dom66.de>
parents: 164
diff changeset
324 pass