Mercurial > hgrepos > Python > apps > py-cutils

annotate _postprocess-sdist.py @ 284:b65d25882e44

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links. A standard sdist contains no symlinks but dereferences symlinks encountered in the source tree.
author Franz Glasner <fzglas.hg@dom66.de>
date Sun, 23 Feb 2025 21:27:48 +0100
parents
children 39a19c008708
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
284
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
1 # -*- coding: utf-8 -*-
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
2 """Postprogress a .tar-sdist to include tests/data with symlinks as symlinks
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
3
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
4 """
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
5
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
6 from __future__ import print_function, absolute_import
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
7
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
8 import ast
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
9 try:
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
10 from configparser import ConfigParser
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
11 except ImportError:
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
12 from ConfigParser import SafeConfigParser as ConfigParser
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
13 import io
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
14 import os
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
15 import tarfile
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
16
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
17 import cutils
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
18 import cutils.util.walk
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
19
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
20
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
21 def main():
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
22 with io.open("setup.cfg", "rt", encoding="utf-8") as cfgfile:
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
23 cp = ConfigParser()
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
24 if hasattr(cp, "read_file"):
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
25 cp.read_file(cfgfile, "setup.cfg")
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
26 else:
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
27 cp.readfp(cfgfile, "setup.cfg")
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
28 project_name = cp.get("metadata", "name")
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
29 project_version = cp.get("metadata", "version")
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
30 if project_version.startswith("attr:"):
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
31 assert project_version == "attr: cutils.__version__"
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
32 project_version = ast.literal_eval(repr(cutils.__version__))
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
33 #
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
34 # Compressed tar files cannot be modified by Python: make sure the
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
35 # originally generated archive is uncompressed.
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
36 #
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
37 assert cp.get("sdist", "formats") == "tar"
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
38
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
39 archive_name = "{}-{}.tar".format(project_name, project_version)
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
40 archive_path = "dist/" + archive_name
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
41 assert os.path.isfile(archive_path)
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
42
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
43 # the directory within the archive
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
44 archive_path_prefix = "{}-{}".format(project_name, project_version)
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
45
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
46 egg_directory = "{}.egg-info".format(project_name.replace("-", "_"))
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
47 assert os.path.isdir(egg_directory)
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
48 sources_txt_path = "{}/SOURCES.txt".format(egg_directory)
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
49 sources_txt_arcname = "{}/{}/SOURCES.txt".format(
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
50 archive_path_prefix,
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
51 egg_directory)
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
52
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
53 with tarfile.TarFile(archive_path, "r") as tf:
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
54 sf = tf.extractfile(sources_txt_arcname)
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
55 try:
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
56 sources_txt = sf.read()
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
57 finally:
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
58 sf.close()
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
59
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
60 with tarfile.TarFile(archive_path, "a") as tf:
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
61 arcname = "{}/tests/data".format(archive_path_prefix)
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
62 try:
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
63 info = tf.getmember(arcname)
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
64 except KeyError:
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
65 pass
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
66 else:
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
67 raise RuntimeError("already postprocessed")
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
68 pre_names = set(tf.getnames())
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
69 tf.add("tests/data", arcname=arcname, recursive=True)
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
70
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
71 #
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
72 # Determine the new files and symlinks that are to be added
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
73 # to SOURCES.txt. Skip directories.
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
74 #
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
75 post_names = set(tf.getnames())
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
76 new_names = list(post_names - pre_names)
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
77 new_names.sort()
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
78 new_sources = []
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
79
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
80 for np in new_names:
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
81 nn = np[len(archive_path_prefix)+1:]
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
82 info = tf.getmember(np)
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
83 if not info.isdir():
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
84 new_sources.append(nn)
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
85
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
86 # Augment SOURCES.txt and add it to the archive
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
87 sources_info = tf.gettarinfo(
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
88 sources_txt_path, arcname=sources_txt_arcname)
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
89 sf = io.BytesIO()
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
90 sf.write(sources_txt)
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
91 if not sources_txt.endswith(b'\n'):
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
92 sf.write(b'\n')
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
93 sf.write(b('\n'.join(new_sources)))
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
94 sources_info.size = len(sf.getvalue())
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
95 sf.seek(0)
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
96 tf.addfile(sources_info, sf)
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
97
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
98
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
99 def b(buf, encoding="ascii"):
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
100 if isinstance(buf, bytes):
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
101 return buf
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
102 else:
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
103 return buf.encode(encoding)
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
104
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
105
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
106 if __name__ == "__main__":
b65d25882e44 REFACTOR: sdist generation: postprocess an sdist to include symbolic links as symbolic links.
Franz Glasner <fzglas.hg@dom66.de>
parents:
diff changeset
107 main()