Python2/PyMuPDF: pipcl.py comparison

comparison pipcl.py @ 41:71bcc18e306f

MERGE: New upstream PyMuPDF v1.26.5 including MuPDF v1.26.10 BUGS: Needs some additional changes yet. Not yet tested.

author	Franz Glasner <fzglas.hg@dom66.de>
date	Sat, 11 Oct 2025 15:24:40 +0200
parents	c4daa0c83d64 a6bc019ac0b2
children	202a1f38a622

comparison

equal deleted inserted replaced

-:8934ac156ef5
+:71bcc18e306f
 '''
 Python packaging operations, including PEP-517 support, for use by a `setup.py`
 script.
-The intention is to take care of as many packaging details as possible so that
+Overview:
-setup.py contains only project-specific information, while also giving as much
-flexibility as possible.
+The intention is to take care of as many packaging details as possible so
+that setup.py contains only project-specific information, while also giving
-For example we provide a function `build_extension()` that can be used to build
+as much flexibility as possible.
-a SWIG extension, but we also give access to the located compiler/linker so
-that a `setup.py` script can take over the details itself.
+For example we provide a function `build_extension()` that can be used
+to build a SWIG extension, but we also give access to the located
-Run doctests with: `python -m doctest pipcl.py`
+compiler/linker so that a `setup.py` script can take over the details
+itself.
-For Graal we require that PIPCL_GRAAL_PYTHON is set to non-graal Python (we
-build for non-graal except with Graal Python's include paths and library
+Doctests:
-directory).
+Doctest strings are provided in some comments.
+Test in the usual way with:
+python -m doctest pipcl.py
+Test specific functions/classes with:
+python pipcl.py --doctest run_if ...
+If no functions or classes are specified, this tests everything.
+Graal:
+For Graal we require that PIPCL_GRAAL_PYTHON is set to non-graal Python (we
+build for non-graal except with Graal Python's include paths and library
+directory).
 '''
 import base64
 import codecs
+import difflib
 import glob
 import hashlib
 import inspect
 import io
 import os
 We also support basic command line handling for use
 with a legacy (pre-PEP-517) pip, as implemented
 by legacy distutils/setuptools and described in:
 https://pip.pypa.io/en/stable/reference/build-system/setup-py/
+The file pyproject.toml must exist; this is checked if/when fn_build() is
+called.
 Here is a `doctest` example of using pipcl to create a SWIG extension
 module. Requires `swig`.
 Create an empty test directory:
 wheel_compression = zipfile.ZIP_DEFLATED,
 wheel_compresslevel = None,
 ):
 '''
-The initial args before `root` define the package
+The initial args before `entry_points` define the
-metadata and closely follow the definitions in:
+package metadata and closely follow the definitions in:
 https://packaging.python.org/specifications/core-metadata/
 Args:
 name:
+Used for metadata `Name`.
 A string, the name of the Python package.
 version:
+Used for metadata `Version`.
 A string, the version of the Python package. Also see PEP-440
 `Version Identification and Dependency Specification`.
 platform:
+Used for metadata `Platform`.
 A string or list of strings.
 supported_platform:
+Used for metadata `Supported-Platform`.
 A string or list of strings.
 summary:
+Used for metadata `Summary`.
 A string, short description of the package.
 description:
+Used for metadata `Description`.
 A string. If contains newlines, a detailed description of the
 package. Otherwise the path of a file containing the detailed
 description of the package.
 description_content_type:
+Used for metadata `Description-Content-Type`.
 A string describing markup of `description` arg. For example
 `text/markdown; variant=GFM`.
 keywords:
+Used for metadata `Keywords`.
 A string containing comma-separated keywords.
 home_page:
+Used for metadata `Home-page`.
 URL of home page.
 download_url:
+Used for metadata `Download-URL`.
 Where this version can be downloaded from.
 author:
+Used for metadata `Author`.
 Author.
 author_email:
+Used for metadata `Author-email`.
 Author email.
 maintainer:
+Used for metadata `Maintainer`.
 Maintainer.
 maintainer_email:
+Used for metadata `Maintainer-email`.
 Maintainer email.
 license:
+Used for metadata `License`.
 A string containing the license text. Written into metadata
 file `COPYING`. Is also written into metadata itself if not
 multi-line.
 classifier:
+Used for metadata `Classifier`.
 A string or list of strings. Also see:
 * https://pypi.org/pypi?%3Aaction=list_classifiers
 * https://pypi.org/classifiers/
 requires_dist:
-A string or list of strings. None items are ignored. Also see PEP-508.
+Used for metadata `Requires-Dist`.
+A string or list of strings, Python packages required
+at runtime. None items are ignored.
 requires_python:
+Used for metadata `Requires-Python`.
 A string or list of strings.
 requires_external:
+Used for metadata `Requires-External`.
 A string or list of strings.
 project_url:
-A string or list of strings, each of the form: `{name}, {url}`.
+Used for metadata `Project-URL`.
+A string or list of strings, each of the form: `{name},
+{url}`.
 provides_extra:
+Used for metadata `Provides-Extra`.
 A string or list of strings.
 entry_points:
 String or dict specifying *.dist-info/entry_points.txt, for
 example:
 be the path to a file; a relative path is treated as relative
 to `root`. If a `bytes` it is the contents of the file to be
 added.
 `to_` identifies what the file should be called within a wheel
-or when installing. If `to_` ends with `/`, the leaf of `from_`
+or when installing. If `to_` is empty or `/` we set it to the
-is appended to it (and `from_` must not be a `bytes`).
+leaf of `from_` (`from_` must not be a `bytes`) - i.e. we place
+the file in the root directory of the wheel; otherwise if
+`to_` ends with `/` the leaf of `from_` is appended to it (and
+`from_` must not be a `bytes`).
 Initial `$dist-info/` in `_to` is replaced by
 `{name}-{version}.dist-info/`; this is useful for license files
 etc.
 we copy `from_` to `{sitepackages}/{to_}`, where
 `sitepackages` is the installation directory, the
 default being `sysconfig.get_path('platlib')` e.g.
 `myvenv/lib/python3.9/site-packages/`.
+When calling this function, we assert that the file
+pyproject.toml exists in the current directory. (We do this
+here rather than in pipcl.Package's constructor, as otherwise
+importing setup.py from non-package-related code could fail.)
 fn_clean:
 A function taking a single arg `all_` that cleans generated
 files. `all_` is true iff `--all` is in argv.
 For safety and convenience, can also returns a list of
 as returned by `fn_build`.
 It can be convenient to use `pipcl.git_items()`.
 The specification for sdists requires that the list contains
-`pyproject.toml`; we enforce this with a diagnostic rather than
+`pyproject.toml`; we enforce this with a Python assert.
-raising an exception, to allow legacy command-line usage.
 tag_python:
 First element of wheel tag defined in PEP-425. If None we use
 `cp{version}`.
 assert_str_or_multi( requires_dist)
 assert_str( requires_python)
 assert_str_or_multi( requires_external)
 assert_str_or_multi( project_url)
 assert_str_or_multi( provides_extra)
+assert re.match('^([A-Z0-9]|[A-Z0-9][A-Z0-9._-]*[A-Z0-9])\\Z', name, re.IGNORECASE), (
+f'Invalid package name'
+f' (https://packaging.python.org/en/latest/specifications/name-normalization/)'
+f': {name!r}'
+)
 # https://packaging.python.org/en/latest/specifications/core-metadata/.
 assert re.match('([A-Z0-9]|[A-Z0-9][A-Z0-9._-]*[A-Z0-9])$', name, re.IGNORECASE), \
 f'Bad name: {name!r}'
 f' wheel_directory={wheel_directory!r}'
 f' config_settings={config_settings!r}'
 f' metadata_directory={metadata_directory!r}'
 )
-if sys.implementation.name == 'graalpy':
+if os.environ.get('CIBUILDWHEEL') == '1':
+# Don't special-case graal builds when running under cibuildwheel.
+pass
+elif sys.implementation.name == 'graalpy':
 # We build for Graal by building a native Python wheel with Graal
 # Python's include paths and library directory. We then rename the
 # wheel to contain graal's tag etc.
 #
 log0(f'### Graal build: deferring to cpython.')
 if inspect.signature(self.fn_sdist).parameters:
 items = self.fn_sdist(config_settings)
 else:
 items = self.fn_sdist()
-prefix = f'{_normalise(self.name)}-{self.version}'
+prefix = f'{_normalise2(self.name)}-{self.version}'
 os.makedirs(sdist_directory, exist_ok=True)
 tarpath = f'{sdist_directory}/{prefix}.tar.gz'
 log2(f'Creating sdist: {tarpath}')
 with tarfile.open(tarpath, 'w:gz') as tar:
 if from_.startswith(f'{os.path.abspath(sdist_directory)}/'):
 # Source files should not be inside <sdist_directory>.
 assert 0, f'Path is inside sdist_directory={sdist_directory}: {from_!r}'
 assert os.path.exists(from_), f'Path does not exist: {from_!r}'
 assert os.path.isfile(from_), f'Path is not a file: {from_!r}'
-if to_rel == 'pyproject.toml':
-found_pyproject_toml = True
 add(from_, to_rel)
+if to_rel == 'pyproject.toml':
-if not found_pyproject_toml:
+found_pyproject_toml = True
-log0(f'Warning: no pyproject.toml specified.')
+assert found_pyproject_toml, f'Cannot create sdist because file not specified: pyproject.toml'
 # Always add a PKG-INFO file.
 add_string(self._metainfo(), 'PKG-INFO')
 if self.license:
 def tag_python(self):
 '''
 Get two-digit python version, e.g. 'cp3.8' for python-3.8.6.
 '''
 if self.tag_python_:
-return self.tag_python_
+ret = self.tag_python_
 else:
-return 'cp' + ''.join(platform.python_version().split('.')[:2])
+ret = 'cp' + ''.join(platform.python_version().split('.')[:2])
+assert '-' not in ret
+return ret
 def tag_abi(self):
 '''
 ABI tag.
 '''
 ret2 = f'{m.group(1)}_0{m.group(2)}'
 log0(f'After macos patch, changing from {ret!r} to {ret2!r}.')
 ret = ret2
 log0( f'tag_platform(): returning {ret=}.')
+assert '-' not in ret
 return ret
 def wheel_name(self):
-return f'{_normalise(self.name)}-{self.version}-{self.tag_python()}-{self.tag_abi()}-{self.tag_platform()}.whl'
+ret = f'{_normalise2(self.name)}-{self.version}-{self.tag_python()}-{self.tag_abi()}-{self.tag_platform()}.whl'
+assert ret.count('-') == 4, f'Expected 4 dash characters in {ret=}.'
+return ret
 def wheel_name_match(self, wheel):
 '''
 Returns true if `wheel` matches our wheel. We basically require the
 name to be the same, except that we accept platform tags that contain
 # This wheel uses Python stable ABI same or older than ours, so
 # we can use it.
 log2(f'py_limited_api; {tag_python=} compatible with {self.tag_python()=}.')
 py_limited_api_compatible = True
-log2(f'{_normalise(self.name) == name=}')
+log2(f'{_normalise2(self.name) == name=}')
 log2(f'{self.version == version=}')
 log2(f'{self.tag_python() == tag_python=} {self.tag_python()=} {tag_python=}')
 log2(f'{py_limited_api_compatible=}')
 log2(f'{self.tag_abi() == tag_abi=}')
 log2(f'{self.tag_platform() in tag_platform.split(".")=}')
 log2(f'{self.tag_platform()=}')
 log2(f'{tag_platform.split(".")=}')
 ret = (1
-and _normalise(self.name) == name
+and _normalise2(self.name) == name
 and self.version == version
 and (self.tag_python() == tag_python or py_limited_api_compatible)
 and self.tag_abi() == tag_abi
 and self.tag_platform() in tag_platform.split('.')
 )
 ret += f'{value}\n'
 return ret
 def _call_fn_build( self, config_settings=None):
 assert self.fn_build
+assert os.path.isfile('pyproject.toml'), (
+'Cannot create package because file does not exist: pyproject.toml'
+)
 log2(f'calling self.fn_build={self.fn_build}')
 if inspect.signature(self.fn_build).parameters:
 ret = self.fn_build(config_settings)
 else:
 ret = self.fn_build()
 assert isinstance( ret, (list, tuple)), \
 f'Expected list/tuple from {self.fn_build} but got: {ret!r}'
+# Check that any extensions that we have built, have same
+# py_limited_api value. If package is marked with py_limited_api=True
+# then non-py_limited_api extensions seem to fail at runtime on
+# Windows.
+#
+# (We could possibly allow package py_limited_api=False and extensions
+# py_limited_api=True, but haven't tested this, and it seems simpler to
+# be strict.)
+for item in ret:
+from_, (to_abs, to_rel) = self._fromto(item)
+from_abs = os.path.abspath(from_)
+is_py_limited_api = _extensions_to_py_limited_api.get(from_abs)
+if is_py_limited_api is not None:
+assert bool(self.py_limited_api) == bool(is_py_limited_api), (
+f'Extension was built with'
+f' py_limited_api={is_py_limited_api} but pipcl.Package'
+f' name={self.name!r} has'
+f' py_limited_api={self.py_limited_api}:'
+f' {from_abs!r}'
+)
 return ret
 def _argv_clean(self, all_):
 '''
 Called by `handle_argv()`. There doesn't seem to be any documentation
 for `setup.py dist_info`, but it appears to be like `egg_info` except
 it writes to a slightly different directory.
 '''
 if root is None:
-root = f'{self.name}-{self.version}.dist-info'
+root = f'{normalise2(self.name)}-{self.version}.dist-info'
 self._write_info(f'{root}/METADATA')
 if self.license:
 with open( f'{root}/COPYING', 'w') as f:
 f.write( self.license)
 f' tag_platform={self.tag_platform_!r}'
 '}'
 )
 def _dist_info_dir( self):
-return f'{_normalise(self.name)}-{self.version}.dist-info'
+return f'{_normalise2(self.name)}-{self.version}.dist-info'
 def _metainfo(self):
 '''
 Returns text for `.egg-info/PKG-INFO` file, or `PKG-INFO` in an sdist
 `.tar.gz` file, or `...dist-info/METADATA` in a wheel.
 If `p` is a string we convert to `(p, p)`. Otherwise we assert that
 `p` is a tuple `(from_, to_)` where `from_` is str/bytes and `to_` is
 str. If `from_` is a bytes it is contents of file to add, otherwise the
 path of an existing file; non-absolute paths are assumed to be relative
-to `self.root`. If `to_` is empty or ends with `/`, we append the leaf
+to `self.root`.
-of `from_` (which must be a str).
+If `to_` is empty or `/` we set it to the leaf of `from_` (which must
+be a str) - i.e. we place the file in the root directory of the wheel;
+otherwise if `to_` ends with `/` we append the leaf of `from_` (which
+must be a str).
 If `to_` starts with `$dist-info/`, we replace this with
 `self._dist_info_dir()`.
 If `to_` starts with `$data/`, we replace this with
 assert isinstance(p, tuple) and len(p) == 2
 from_, to_ = p
 assert isinstance(from_, (str, bytes))
 assert isinstance(to_, str)
-if to_.endswith('/') or to_=='':
+if to_ == '/' or to_ == '':
+to_ = os.path.basename(from_)
+elif to_.endswith('/'):
 to_ += os.path.basename(from_)
 prefix = '$dist-info/'
 if to_.startswith( prefix):
 to_ = f'{self._dist_info_dir()}/{to_[ len(prefix):]}'
 prefix = '$data/'
 if to_.startswith( prefix):
-to_ = f'{self.name}-{self.version}.data/{to_[ len(prefix):]}'
+to_ = f'{_normalise2(self.name)}-{self.version}.data/{to_[ len(prefix):]}'
 if isinstance(from_, str):
 from_, _, _ = self._path_relative_to_root( from_, assert_within_root=False, resolve_symlinks=resolve_symlinks)
 to_ = self._path_relative_to_root(to_, resolve_symlinks=resolve_symlinks)
 assert isinstance(from_, (str, bytes))
 log2(f'returning {from_=} {to_=}')
 return from_, to_
+_extensions_to_py_limited_api = dict()
 def build_extension(
 name,
 path_i,
 outdir,
+*,
 builddir=None,
 includes=None,
 defines=None,
 libpaths=None,
 libs=None,
 debug=False,
 compiler_extra='',
 linker_extra='',
 swig=None,
 cpp=True,
+source_extra=None,
 prerequisites_swig=None,
 prerequisites_compile=None,
 prerequisites_link=None,
 infer_swig_includes=True,
 py_limited_api=False,
 `/LIBPATH:` on Windows or `-L` on Unix.
 libs
 A string, or a sequence of library names. Each item is prefixed
 with `-l` on non-Windows.
 optimise:
-Whether to use compiler optimisations.
+Whether to use compiler optimisations and define NDEBUG.
 debug:
 Whether to build with debug symbols.
 compiler_extra:
 Extra compiler flags. Can be None.
 linker_extra:
 Extra linker flags. Can be None.
 swig:
 Swig command; if false we use 'swig'.
 cpp:
 If true we tell SWIG to generate C++ code instead of C.
+source_extra:
+Extra source files to build into the shared library,
 prerequisites_swig:
 prerequisites_compile:
 prerequisites_link:
 [These are mainly for use on Windows. On other systems we
 infer_swig_includes:
 If true, we extract `-I<path>` and `-I <path>` args from
 `compile_extra` (also `/I` on windows) and use them with swig so
 that it can see the same header files as C/C++. This is useful
 when using enviromment variables such as `CC` and `CXX` to set
-`compile_extra.
+`compile_extra`.
 py_limited_api:
 If true we build for current Python's limited API / stable ABI.
+Note that we will assert false if this extension is added to a
+pipcl.Package that has a different <py_limited_api>, because
+on Windows importing a non-py_limited_api extension inside a
+py_limited=True package fails.
 Returns the leafname of the generated library file within `outdir`, e.g.
 `_{name}.so` on Unix or `_{name}.cp311-win_amd64.pyd` on Windows.
 '''
 if compiler_extra is None:
 linker_extra = ''
 if builddir is None:
 builddir = outdir
 if not swig:
 swig = 'swig'
+if source_extra is None:
+source_extra = list()
+if isinstance(source_extra, str):
+source_extra = [source_extra]
 includes_text = _flags( includes, '-I')
 defines_text = _flags( defines, '-D')
 libpaths_text = _flags( libpaths, '/LIBPATH:', '"') if windows() else _flags( libpaths, '-L')
 libs_text = _flags( libs, '' if windows() else '-l')
 path_cpp = f'{builddir}/{os.path.basename(path_i)}'
 path_cpp += '.cpp' if cpp else '.c'
 os.makedirs( outdir, exist_ok=True)
 # Run SWIG.
+#
 if infer_swig_includes:
 # Extract include flags from `compiler_extra`.
 swig_includes_extra = ''
-compiler_extra_items = compiler_extra.split()
+compiler_extra_items = shlex.split(compiler_extra)
 i = 0
 while i < len(compiler_extra_items):
 item = compiler_extra_items[i]
 # Swig doesn't seem to like a space after `I`.
 if item == '-I' or (windows() and item == '/I'):
 path_i,
 prerequisites_swig,
 prerequisites_swig2,
 )
-so_suffix = _so_suffix(use_so_versioning = not py_limited_api)
+if pyodide():
+so_suffix = '.so'
+log0(f'pyodide: PEP-3149 suffix untested, so omitting. {_so_suffix()=}.')
+else:
+so_suffix = _so_suffix(use_so_versioning = not py_limited_api)
 path_so_leaf = f'_{name}{so_suffix}'
 path_so = f'{outdir}/{path_so_leaf}'
 py_limited_api2 = current_py_limited_api() if py_limited_api else None
+compiler_command, pythonflags = base_compiler(cpp=cpp)
+linker_command, _ = base_linker(cpp=cpp)
+# setuptools on Linux seems to use slightly different compile flags:
+#
+# -fwrapv -O3 -Wall -O2 -g0 -DPY_CALL_TRAMPOLINE
+#
+general_flags = ''
 if windows():
-path_obj = f'{path_so}.obj'
 permissive = '/permissive-'
 EHsc = '/EHsc'
 T = '/Tp' if cpp else '/Tc'
 optimise2 = '/DNDEBUG /O2' if optimise else '/D_DEBUG'
-debug2 = ''
+debug2 = '/Zi' if debug else ''
+py_limited_api3 = f'/DPy_LIMITED_API={py_limited_api2}' if py_limited_api2 else ''
+else:
 if debug:
-debug2 = '/Zi'  # Generate .pdb.
+general_flags += '/Zi' if windows() else ' -g'
-# debug2 = '/Z7'    # Embed debug info in .obj files.
+if optimise:
+general_flags += ' /DNDEBUG /O2' if windows() else ' -O2 -DNDEBUG'
-py_limited_api3 = f'/DPy_LIMITED_API={py_limited_api2}' if py_limited_api2 else ''
+py_limited_api3 = f'-DPy_LIMITED_API={py_limited_api2}' if py_limited_api2 else ''
-# As of 2023-08-23, it looks like VS tools create slightly
-# .dll's each time, even with identical inputs.
+if windows():
+pass
+elif darwin():
+# MacOS's linker does not like `-z origin`.
+rpath_flag = "-Wl,-rpath,@loader_path/"
+# Avoid `Undefined symbols for ... "_PyArg_UnpackTuple" ...'.
+general_flags += ' -undefined dynamic_lookup'
+elif pyodide():
+# Setting `-Wl,-rpath,'$ORIGIN',-z,origin` gives:
+#   emcc: warning: ignoring unsupported linker flag: `-rpath` [-Wlinkflags]
+#   wasm-ld: error: unknown -z value: origin
 #
-# Some info about this is at:
+rpath_flag = "-Wl,-rpath,'$ORIGIN'"
-# https://nikhilism.com/post/2020/windows-deterministic-builds/.
+else:
-# E.g. an undocumented linker flag `/Brepro`.
+rpath_flag = "-Wl,-rpath,'$ORIGIN',-z,origin"
-#
+# Fun fact - on Linux, if the -L and -l options are before '{path_cpp}'
-command, pythonflags = base_compiler(cpp=cpp)
+# they seem to be ignored...
-command = f'''
+#
-{command}
+path_os = list()
-# General:
-/c                          # Compiles without linking.
+for path_source in [path_cpp] + source_extra:
-{EHsc}                      # Enable "Standard C++ exception handling".
+path_o = f'{path_source}.obj' if windows() else f'{path_source}.o'
+path_os.append(f' {path_o}')
-#/MD                         # Creates a multithreaded DLL using MSVCRT.lib.
-{'/MDd' if debug else '/MD'}
+prerequisites_path = f'{path_o}.d'
-# Input/output files:
+if windows():
-{T}{path_cpp}               # /Tp specifies C++ source file.
+compiler_command2 = f'''
-/Fo{path_obj}               # Output file. codespell:ignore
+{compiler_command}
+# General:
-# Include paths:
+/c                          # Compiles without linking.
-{includes_text}
+{EHsc}                      # Enable "Standard C++ exception handling".
-{pythonflags.includes}      # Include path for Python headers.
+#/MD                         # Creates a multithreaded DLL using MSVCRT.lib.
-# Code generation:
+{'/MDd' if debug else '/MD'}
-{optimise2}
-{debug2}
+# Input/output files:
-{permissive}                # Set standard-conformance mode.
+{T}{path_source}            # /Tp specifies C++ source file.
+/Fo{path_o}                 # Output file. codespell:ignore
-# Diagnostics:
-#/FC                         # Display full path of source code files passed to cl.exe in diagnostic text.
+# Include paths:
-/W3                         # Sets which warning level to output. /W3 is IDE default.
+{includes_text}
-/diagnostics:caret          # Controls the format of diagnostic messages.
+{pythonflags.includes}      # Include path for Python headers.
-/nologo                     #
+# Code generation:
-{defines_text}
+{optimise2}
-{compiler_extra}
+{debug2}
+{permissive}                # Set standard-conformance mode.
-{py_limited_api3}
-'''
+# Diagnostics:
-run_if( command, path_obj, path_cpp, prerequisites_compile)
+#/FC                         # Display full path of source code files passed to cl.exe in diagnostic text.
+/W3                         # Sets which warning level to output. /W3 is IDE default.
-command, pythonflags = base_linker(cpp=cpp)
+/diagnostics:caret          # Controls the format of diagnostic messages.
+/nologo                     #
+{defines_text}
+{compiler_extra}
+{py_limited_api3}
+'''
+else:
+compiler_command2 = f'''
+{compiler_command}
+-fPIC
+{general_flags.strip()}
+{pythonflags.includes}
+{includes_text}
+{defines_text}
+-MD -MF {prerequisites_path}
+-c {path_source}
+-o {path_o}
+{compiler_extra}
+{py_limited_api3}
+'''
+run_if(
+compiler_command2,
+path_o,
+path_source,
+[path_source] + _get_prerequisites(prerequisites_path),
+)
+# Link
+prerequisites_path = f'{path_so}.d'
+if windows():
 debug2 = '/DEBUG' if debug else ''
 base, _ = os.path.splitext(path_so_leaf)
-command = f'''
+command2 = f'''
-{command}
+{linker_command}
 /DLL                    # Builds a DLL.
 /EXPORT:PyInit__{name}  # Exports a function.
 /IMPLIB:{base}.lib      # Overrides the default import library name.
 {libpaths_text}
 {pythonflags.ldflags}
 /OUT:{path_so}          # Specifies the output file name.
 {debug2}
 /nologo
 {libs_text}
-{path_obj}
+{' '.join(path_os)}
 {linker_extra}
 '''
-run_if( command, path_so, path_obj, prerequisites_link)
-else:
-# Not Windows.
-#
-command, pythonflags = base_compiler(cpp=cpp)
-# setuptools on Linux seems to use slightly different compile flags:
-#
-# -fwrapv -O3 -Wall -O2 -g0 -DPY_CALL_TRAMPOLINE
-#
-general_flags = ''
-if debug:
-general_flags += ' -g'
-if optimise:
-general_flags += ' -O2 -DNDEBUG'
 if os.environ.get('EXTRA_CHECKS', '1') != '0':
 general_flags += ' -fno-delete-null-pointer-checks'
 general_flags += ' -Werror=implicit-function-declaration'
 general_flags += ' -fstack-clash-protection'
 general_flags += ' -fstack-protector-strong'
+elif pyodide():
-py_limited_api3 = f'-DPy_LIMITED_API={py_limited_api2}' if py_limited_api2 else ''
+command2 = f'''
+{linker_command}
-if darwin():
+-MD -MF {prerequisites_path}
-# MacOS's linker does not like `-z origin`.
+-o {path_so}
-rpath_flag = "-Wl,-rpath,@loader_path/"
+{' '.join(path_os)}
+{libpaths_text}
-# Avoid `Undefined symbols for ... "_PyArg_UnpackTuple" ...'.
+{libs_text}
-general_flags += ' -undefined dynamic_lookup'
+{linker_extra}
-elif pyodide():
+{pythonflags.ldflags}
-# Setting `-Wl,-rpath,'$ORIGIN',-z,origin` gives:
+{rpath_flag}
-#   emcc: warning: ignoring unsupported linker flag: `-rpath` [-Wlinkflags]
+'''
-#   wasm-ld: error: unknown -z value: origin
+else:
-#
+command2 = f'''
-log0(f'pyodide: PEP-3149 suffix untested, so omitting. {_so_suffix()=}.')
+{linker_command}
-path_so_leaf = f'_{name}.so'
+-shared
-path_so = f'{outdir}/{path_so_leaf}'
+{general_flags.strip()}
+-MD -MF {prerequisites_path}
-rpath_flag = ''
+-o {path_so}
-else:
+{' '.join(path_os)}
-rpath_flag = "-Wl,-rpath,'$ORIGIN',-z,origin"
+{libpaths_text}
-path_so = f'{outdir}/{path_so_leaf}'
+{libs_text}
-# Fun fact - on Linux, if the -L and -l options are before '{path_cpp}'
+{linker_extra}
-# they seem to be ignored...
+{pythonflags.ldflags}
-#
+{rpath_flag}
-prerequisites = list()
+{py_limited_api3}
+'''
-if pyodide():
+link_was_run = run_if(
-# Looks like pyodide's `cc` can't compile and link in one invocation.
+command2,
-prerequisites_compile_path = f'{path_cpp}.o.d'
+path_so,
-prerequisites += _get_prerequisites( prerequisites_compile_path)
+path_cpp,
-command = f'''
+*path_os,
-{command}
+*_get_prerequisites(f'{path_so}.d'),
--fPIC
+)
-{general_flags.strip()}
-{pythonflags.includes}
+if link_was_run and darwin():
-{includes_text}
+# We need to patch up references to shared libraries in `libs`.
-{defines_text}
+sublibraries = list()
--MD -MF {prerequisites_compile_path}
+for lib in () if libs is None else libs:
--c {path_cpp}
+for libpath in libpaths:
--o {path_cpp}.o
+found = list()
-{compiler_extra}
+for suffix in '.so', '.dylib':
-{py_limited_api3}
+path = f'{libpath}/lib{os.path.basename(lib)}{suffix}'
-'''
+if os.path.exists( path):
-prerequisites_link_path = f'{path_cpp}.o.d'
+found.append( path)
-prerequisites += _get_prerequisites( prerequisites_link_path)
+if found:
-ld, _ = base_linker(cpp=cpp)
+assert len(found) == 1, f'More than one file matches lib={lib!r}: {found}'
-command += f'''
+sublibraries.append( found[0])
-&& {ld}
+break
-{path_cpp}.o
+else:
--o {path_so}
+log2(f'Warning: can not find path of lib={lib!r} in libpaths={libpaths}')
--MD -MF {prerequisites_link_path}
+macos_patch( path_so, *sublibraries)
-{rpath_flag}
-{libpaths_text}
-{libs_text}
-{linker_extra}
-{pythonflags.ldflags}
-'''
-else:
-# We use compiler to compile and link in one command.
-prerequisites_path = f'{path_so}.d'
-prerequisites = _get_prerequisites(prerequisites_path)
-command = f'''
-{command}
--fPIC
--shared
-{general_flags.strip()}
-{pythonflags.includes}
-{includes_text}
-{defines_text}
-{path_cpp}
--MD -MF {prerequisites_path}
--o {path_so}
-{compiler_extra}
-{libpaths_text}
-{linker_extra}
-{pythonflags.ldflags}
-{libs_text}
-{rpath_flag}
-{py_limited_api3}
-'''
-command_was_run = run_if(
-command,
-path_so,
-path_cpp,
-prerequisites_compile,
-prerequisites_link,
-prerequisites,
-)
-if command_was_run and darwin():
-# We need to patch up references to shared libraries in `libs`.
-sublibraries = list()
-for lib in () if libs is None else libs:
-for libpath in libpaths:
-found = list()
-for suffix in '.so', '.dylib':
-path = f'{libpath}/lib{os.path.basename(lib)}{suffix}'
-if os.path.exists( path):
-found.append( path)
-if found:
-assert len(found) == 1, f'More than one file matches lib={lib!r}: {found}'
-sublibraries.append( found[0])
-break
-else:
-log2(f'Warning: can not find path of lib={lib!r} in libpaths={libpaths}')
-macos_patch( path_so, *sublibraries)
 #run(f'ls -l {path_so}', check=0)
 #run(f'file {path_so}', check=0)
+_extensions_to_py_limited_api[os.path.abspath(path_so)] = py_limited_api
 return path_so_leaf
 # Functions that might be useful.
 #
 capture=1,
 check=0
 )
 if not e:
 branch = out.strip()
-log(f'git_info(): directory={directory!r} returning branch={branch!r} sha={sha!r} comment={comment!r}')
+log1(f'git_info(): directory={directory!r} returning branch={branch!r} sha={sha!r} comment={comment!r}')
 return sha, comment, diff, branch
 def git_items( directory, submodules=False):
 '''
 ret.append(path)
 return ret
 def git_get(
-remote,
 local,
 *,
+remote=None,
 branch=None,
+tag=None,
+text=None,
 depth=1,
 env_extra=None,
-tag=None,
 update=True,
 submodules=True,
-default_remote=None,
 ):
 '''
-Ensures that <local> is a git checkout (at either <tag>, or <branch> HEAD)
+Creates/updates local checkout <local> of remote repository and returns
-of a remote repository.
+absolute path of <local>.
-Exactly one of <branch> and <tag> must be specified, or <remote> must start
+If <text> is set but does not start with 'git:', it is assumed to be an up
-with 'git:' and match the syntax described below.
+to date local checkout, and we return absolute path of <text> without doing
+any git operations.
 Args:
+local:
+Local directory. Created and/or updated using `git clone` and `git
+fetch` etc.
 remote:
 Remote git repostitory, for example
-'https://github.com/ArtifexSoftware/mupdf.git'.
+'https://github.com/ArtifexSoftware/mupdf.git'. Can be overridden
+by <text>.
+branch:
+Branch to use; can be overridden by <text>.
+tag:
+Tag to use; can be overridden by <text>.
+text:
+If None or empty:
+Ignored.
-If starts with 'git:', the remaining text should be a command-line
+If starts with 'git:':
-style string containing some or all of these args:
+The remaining text should be a command-line
---branch <branch>
+style string containing some or all of these args:
---tag <tag>
+--branch <branch>
-<remote>
+--tag <tag>
-These overrides <branch>, <tag> and <default_remote>.
+<remote>
+These overrides <branch>, <tag> and <remote>.
+Otherwise:
+<text> is assumed to be a local directory, and we simply return
+it as an absolute path without doing any git operations.
 For example these all clone/update/branch master of https://foo.bar/qwerty.git to local
 checkout 'foo-local':
-git_get('https://foo.bar/qwerty.git', 'foo-local', branch='master')
+git_get('foo-local', remote='https://foo.bar/qwerty.git', branch='master')
-git_get('git:--branch master https://foo.bar/qwerty.git', 'foo-local')
+git_get('foo-local', text='git:--branch master https://foo.bar/qwerty.git')
-git_get('git:--branch master', 'foo-local', default_remote='https://foo.bar/qwerty.git')
+git_get('foo-local', text='git:--branch master', remote='https://foo.bar/qwerty.git')
-git_get('git:', 'foo-local', branch='master', default_remote='https://foo.bar/qwerty.git')
+git_get('foo-local', text='git:', branch='master', remote='https://foo.bar/qwerty.git')
-local:
-Local directory. If <local>/.git exists, we attempt to run `git
-update` in it.
-branch:
-Branch to use. Is used as default if remote starts with 'git:'.
 depth:
 Depth of local checkout when cloning and fetching, or None.
 env_extra:
 Dict of extra name=value environment variables to use whenever we
 run git.
-tag:
-Tag to use. Is used as default if remote starts with 'git:'.
 update:
 If false we do not update existing repository. Might be useful if
 testing without network access.
 submodules:
 If true, we clone with `--recursive --shallow-submodules` and run
 `git submodule update --init --recursive` before returning.
-default_remote:
-The remote URL if <remote> starts with 'git:' but does not specify
-the remote URL.
 '''
 log0(f'{remote=} {local=} {branch=} {tag=}')
-if remote.startswith('git:'):
-remote0 = remote
+if text:
-args = iter(shlex.split(remote0[len('git:'):]))
+if text.startswith('git:'):
-remote = default_remote
+args = iter(shlex.split(text[len('git:'):]))
 while 1:
 try:
 arg = next(args)
 except StopIteration:
 break
 if arg == '--branch':
 branch = next(args)
 tag = None
 elif arg == '--tag':
-tag == next(args)
+tag = next(args)
 branch = None
 else:
 remote = arg
-assert remote, f'{default_remote=} and no remote specified in remote={remote0!r}.'
+assert remote, f'<remote> unset and no remote specified in {text=}.'
-assert branch or tag, f'{branch=} {tag=} and no branch/tag specified in remote={remote0!r}.'
+assert branch or tag, f'<branch> and <tag> unset and no branch/tag specified in {text=}.'
+else:
+log0(f'Using local directory {text!r}.')
+return os.path.abspath(text)
-assert (branch and not tag) or (not branch and tag), f'Must specify exactly one of <branch> and <tag>.'
+assert (branch and not tag) or (not branch and tag), f'Must specify exactly one of <branch> and <tag>; {branch=} {tag=}.'
 depth_arg = f' --depth {depth}' if depth else ''
 def do_update():
 # This seems to pull in the entire repository.
 log0(f'do_update(): attempting to update {local=}.')
 # Remove any local changes.
-run(f'cd {local} && git checkout .', env_extra=env_extra)
+run(f'cd {local} && git reset --hard', env_extra=env_extra)
 if tag:
 # `-u` avoids `fatal: Refusing to fetch into current branch`.
 # Using '+' and `revs/tags/` prefix seems to avoid errors like:
 #   error: cannot update ref 'refs/heads/v3.16.44':
 #   trying to write non-commit object
 if submodules:
 run(f'cd {local} && git submodule update --init --recursive', env_extra=env_extra)
 # Show sha of checkout.
 run( f'cd {local} && git show --pretty=oneline|head -n 1', check=False)
+return os.path.abspath(local)
 def run(
 command,
 *,
 ldflags2 = self.ldflags.replace(' -lcrypt ', ' ')
 if ldflags2 != self.ldflags:
 log2(f'### Have removed `-lcrypt` from ldflags: {self.ldflags!r} -> {ldflags2!r}')
 self.ldflags = ldflags2
-log1(f'{self.includes=}')
+if 0:
-log1(f'    {includes_=}')
+log1(f'{self.includes=}')
-log1(f'{self.ldflags=}')
+log1(f'    {includes_=}')
-log1(f'    {ldflags_=}')
+log1(f'{self.ldflags=}')
+log1(f'    {ldflags_=}')
 def macos_add_cross_flags(command):
 '''
 If running on MacOS and environment variables ARCHFLAGS is set
 '''
 #log(f'sys.maxsize={hex(sys.maxsize)}')
 return f'x{32 if sys.maxsize == 2**31 - 1 else 64}'
-def run_if( command, out, *prerequisites):
+def run_if( command, out, *prerequisites, caller=1):
 '''
 Runs a command only if the output file is not up to date.
 Args:
 command:
 >>> out = 'run_if_test_out'
 >>> if os.path.exists( out):
 ...     os.remove( out)
 >>> if os.path.exists( f'{out}.cmd'):
 ...     os.remove( f'{out}.cmd')
->>> run_if( f'touch {out}', out)
+>>> run_if( f'touch {out}', out, caller=0)
 pipcl.py:run_if(): Running command because: File does not exist: 'run_if_test_out'
 pipcl.py:run_if(): Running: touch run_if_test_out
 True
 If we repeat, the output file will be up to date so the command is not run:
->>> run_if( f'touch {out}', out)
+>>> run_if( f'touch {out}', out, caller=0)
 pipcl.py:run_if(): Not running command because up to date: 'run_if_test_out'
 If we change the command, the command is run:
->>> run_if( f'touch  {out}', out)
+>>> run_if( f'touch {out};', out, caller=0)
-pipcl.py:run_if(): Running command because: Command has changed
+pipcl.py:run_if(): Running command because: Command has changed:
-pipcl.py:run_if(): Running: touch  run_if_test_out
+pipcl.py:run_if():     @@ -1,2 +1,2 @@
+pipcl.py:run_if():      touch
+pipcl.py:run_if():     -run_if_test_out
+pipcl.py:run_if():     +run_if_test_out;
+pipcl.py:run_if():
+pipcl.py:run_if(): Running: touch run_if_test_out;
 True
 If we add a prerequisite that is newer than the output, the command is run:
 >>> time.sleep(1)
 >>> prerequisite = 'run_if_test_prerequisite'
 >>> run( f'touch {prerequisite}', caller=0)
 pipcl.py:run(): Running: touch run_if_test_prerequisite
->>> run_if( f'touch  {out}', out, prerequisite)
+>>> run_if( f'touch  {out}', out, prerequisite, caller=0)
-pipcl.py:run_if(): Running command because: Prerequisite is new: 'run_if_test_prerequisite'
+pipcl.py:run_if(): Running command because: Command has changed:
+pipcl.py:run_if():     @@ -1,2 +1,2 @@
+pipcl.py:run_if():      touch
+pipcl.py:run_if():     -run_if_test_out;
+pipcl.py:run_if():     +run_if_test_out
+pipcl.py:run_if():
 pipcl.py:run_if(): Running: touch  run_if_test_out
 True
 If we repeat, the output will be newer than the prerequisite, so the
 command is not run:
->>> run_if( f'touch  {out}', out, prerequisite)
+>>> run_if( f'touch  {out}', out, prerequisite, caller=0)
 pipcl.py:run_if(): Not running command because up to date: 'run_if_test_out'
 '''
 doit = False
 cmd_path = f'{out}.cmd'
 if os.path.isfile( cmd_path):
 with open( cmd_path) as f:
 cmd = f.read()
 else:
 cmd = None
-if command != cmd:
+cmd_args = shlex.split(cmd or '')
+command_args = shlex.split(command or '')
+if command_args != cmd_args:
 if cmd is None:
 doit = 'No previous command stored'
 else:
 doit = f'Command has changed'
 if 0:
-doit += f': {cmd!r} => {command!r}'
+doit += f':\n    {cmd!r}\n    {command!r}'
+if 0:
+doit += f'\nbefore:\n'
+doit += textwrap.indent(cmd, '    ')
+doit += f'\nafter:\n'
+doit += textwrap.indent(command, '    ')
+if 1:
+# Show diff based on commands split into pseudo lines by
+# shlex.split().
+doit += ':\n'
+lines = difflib.unified_diff(
+cmd.split(),
+command.split(),
+lineterm='',
+)
+# Skip initial lines.
+assert next(lines) == '--- '
+assert next(lines) == '+++ '
+for line in lines:
+doit += f'    {line}\n'
 if not doit:
 # See whether any prerequisites are newer than target.
 def _make_prerequisites(p):
 if isinstance( p, (list, tuple)):
 return [p]
 prerequisites_all = list()
 for p in prerequisites:
 prerequisites_all += _make_prerequisites( p)
 if 0:
-log2( 'prerequisites_all:')
+log2( 'prerequisites_all:', caller=caller+1)
 for i in  prerequisites_all:
-log2( f'    {i!r}')
+log2( f'    {i!r}', caller=caller+1)
 pre_mtime = 0
 pre_path = None
 for prerequisite in prerequisites_all:
 if isinstance( prerequisite, str):
 mtime = _fs_mtime_newest( prerequisite)
 elif prerequisite:
 doit = str(prerequisite)
 break
 if not doit:
 if pre_mtime > out_mtime:
-doit = f'Prerequisite is new: {pre_path!r}'
+doit = f'Prerequisite is new: {os.path.abspath(pre_path)!r}'
 if doit:
 # Remove `cmd_path` before we run the command, so any failure
 # will force rerun next time.
 #
 try:
 os.remove( cmd_path)
 except Exception:
 pass
-log1( f'Running command because: {doit}')
+log1( f'Running command because: {doit}', caller=caller+1)
-run( command)
+run( command, caller=caller+1)
 # Write the command we ran, into `cmd_path`.
 with open( cmd_path, 'w') as f:
 f.write( command)
 return True
 else:
-log1( f'Not running command because up to date: {out!r}')
+log1( f'Not running command because up to date: {out!r}', caller=caller+1)
 if 0:
 log2( f'out_mtime={time.ctime(out_mtime)} pre_mtime={time.ctime(pre_mtime)}.'
 f' pre_path={pre_path!r}: returning {ret!r}.'
 )
 def _normalise(name):
 # https://packaging.python.org/en/latest/specifications/name-normalization/#name-normalization
 return re.sub(r"[-_.]+", "-", name).lower()
+def _normalise2(name):
+# https://packaging.python.org/en/latest/specifications/binary-distribution-format/
+return _normalise(name).replace('-', '_')
 def _assert_version_pep_440(version):
 assert re.match(
 r'^([1-9][0-9]*!)?(0|[1-9][0-9]*)(\.(0|[1-9][0-9]*))*((a|b|rc)(0|[1-9][0-9]*))?(\.post(0|[1-9][0-9]*))?(\.dev(0|[1-9][0-9]*))?(?:\+([a-z0-9]+(?:[-_\.][a-z0-9]+)*))?$',
 version,
 ), \
 '''
 Sets whether to include line numbers; helps with doctest.
 '''
 global g_log_line_numbers
 g_log_line_numbers = bool(yes)
+def log(text='', caller=1):
+_log(text, 0, caller+1)
 def log0(text='', caller=1):
 _log(text, 0, caller+1)
 def log1(text='', caller=1):
 print(f'{filename}:{fr.lineno}:{fr.function}(): {line}', file=sys.stdout, flush=1)
 else:
 print(f'{filename}:{fr.function}(): {line}', file=sys.stdout, flush=1)
-def relpath(path, start=None):
+def relpath(path, start=None, allow_up=True):
 '''
 A safe alternative to os.path.relpath(), avoiding an exception on Windows
 if the drive needs to change - in this case we use os.path.abspath().
+Args:
+path:
+Path to be processed.
+start:
+Start directory or current directory if None.
+allow_up:
+If false we return absolute path is <path> is not within <start>.
 '''
 if windows():
 try:
-return os.path.relpath(path, start)
+ret = os.path.relpath(path, start)
 except ValueError:
 # os.path.relpath() fails if trying to change drives.
-return os.path.abspath(path)
+ret = os.path.abspath(path)
 else:
-return os.path.relpath(path, start)
+ret = os.path.relpath(path, start)
+if not allow_up and ret.startswith('../') or ret.startswith('..\\'):
+ret = os.path.abspath(path)
+return ret
 def _so_suffix(use_so_versioning=True):
 '''
 Filename suffix for shared libraries is defined in pep-3149.  The
 ret = list()
 items = self._items()
 for path, id_ in items.items():
 id0 = self.items0.get(path)
 if id0 != id_:
-#mtime0, hash0 = id0
-#mtime1, hash1 = id_
-#log0(f'New/modified file {path=}.')
-#log0(f'    {mtime0=} {"==" if mtime0==mtime1 else "!="} {mtime1=}.')
-#log0(f'    {hash0=} {"==" if hash0==hash1 else "!="} {hash1=}.')
 ret.append(path)
 return ret
+def get_n(self, n):
+'''
+Returns new files matching <glob_pattern>, asserting that there are
+exactly <n>.
+'''
+ret = self.get()
+assert len(ret) == n, f'{len(ret)=}: {ret}'
+return ret
 def get_one(self):
 '''
 Returns new match of <glob_pattern>, asserting that there is exactly
 one.
 '''
-ret = self.get()
+return self.get_n(1)[0]
-assert len(ret) == 1, f'{len(ret)=}'
-return ret[0]
 def _file_id(self, path):
 mtime = os.stat(path).st_mtime
 with open(path, 'rb') as f:
 content = f.read()
 hash_ = hashlib.md5(content).digest()
 Otherwise we simply return <swig>.
 Args:
 swig:
-If starts with 'git:', passed as <remote> arg to git_remote().
+If starts with 'git:', passed as <text> arg to git_get().
 quick:
 If true, we do not update/build local checkout if the binary is
 already present.
 swig_local:
 path to use for checkout.
 '''
 if swig and swig.startswith('git:'):
-assert platform.system() != 'Windows'
+assert platform.system() != 'Windows', f'Cannot build swig on Windows.'
-swig_local = os.path.abspath(swig_local)
+# Note that {swig_local}/install/bin/swig doesn't work on MacOS because
-# Note that {swig_local}/install/bin/swig doesn't work on MacoS because
 # {swig_local}/INSTALL is a file and the fs is case-insensitive.
 swig_binary = f'{swig_local}/install-dir/bin/swig'
 if quick and os.path.isfile(swig_binary):
 log1(f'{quick=} and {swig_binary=} already exists, so not downloading/building.')
 else:
 # Clone swig.
 swig_env_extra = None
-git_get(
+swig_local = git_get(
-swig,
 swig_local,
-default_remote='https://github.com/swig/swig.git',
+text=swig,
+remote='https://github.com/swig/swig.git',
 branch='master',
 )
 if darwin():
 run(f'brew install automake')
 run(f'brew install pcre2')
 # > parallel can cause all kinds of trouble.
 # >
 # > If you need to have bison first in your PATH, run:
 # >   echo 'export PATH="/opt/homebrew/opt/bison/bin:$PATH"' >> ~/.zshrc
 #
-run(f'brew install bison')
+swig_env_extra = dict()
-PATH = os.environ['PATH']
+macos_add_brew_path('bison', swig_env_extra)
-PATH = f'/opt/homebrew/opt/bison/bin:{PATH}'
+run(f'which bison')
-swig_env_extra = dict(PATH=PATH)
+run(f'which bison', env_extra=swig_env_extra)
 # Build swig.
 run(f'cd {swig_local} && ./autogen.sh', env_extra=swig_env_extra)
 run(f'cd {swig_local} && ./configure --prefix={swig_local}/install-dir', env_extra=swig_env_extra)
 run(f'cd {swig_local} && make', env_extra=swig_env_extra)
 run(f'cd {swig_local} && make install', env_extra=swig_env_extra)
 assert os.path.isfile(swig_binary)
 return swig_binary
 else:
 return swig
+def macos_add_brew_path(package, env=None, gnubin=True):
+'''
+Adds path(s) for Brew <package>'s binaries to env['PATH'].
+Args:
+package:
+Name of package. We get <package_root> of installed package by
+running `brew --prefix <package>`.
+env:
+The environment dict to modify. If None we use os.environ. If PATH
+is not in <env>, we first copy os.environ['PATH'] into <env>.
+gnubin:
+If true, we also add path to gnu binaries if it exists,
+<package_root>/libexe/gnubin.
+'''
+if not darwin():
+return
+if env is None:
+env = os.environ
+if 'PATH' not in env:
+env['PATH'] = os.environ['PATH']
+package_root = run(f'brew --prefix {package}', capture=1).strip()
+def add(path):
+if os.path.isdir(path):
+log1(f'Adding to $PATH: {path}')
+PATH = env['PATH']
+env['PATH'] = f'{path}:{PATH}'
+add(f'{package_root}/bin')
+if gnubin:
+add(f'{package_root}/libexec/gnubin')
 def _show_dict(d):
 ret = ''
 for n in sorted(d.keys()):
 ldflags_ = f'-L {ldflags_}'
 includes_ = ' '.join(includes_)
 return includes_, ldflags_
+def venv_in(path=None):
+'''
+If path is None, returns true if we are in a venv. Otherwise returns true
+only if we are in venv <path>.
+'''
+if path:
+return os.path.abspath(sys.prefix) == os.path.abspath(path)
+else:
+return sys.prefix != sys.base_prefix
+def venv_run(args, path, recreate=True, clean=False):
+'''
+Runs Python command inside venv and returns termination code.
+Args:
+args:
+List of args or string command.
+path:
+Path of venv directory.
+recreate:
+If false we do not run `<sys.executable> -m venv <path>` if <path>
+already exists. This avoids a delay in the common case where <path>
+is already set up, but fails if <path> exists but does not contain
+a valid venv.
+clean:
+If true we first delete <path>.
+'''
+if clean:
+log(f'Removing any existing venv {path}.')
+assert path.startswith('venv-')
+shutil.rmtree(path, ignore_errors=1)
+if recreate or not os.path.isdir(path):
+run(f'{sys.executable} -m venv {path}')
+if isinstance(args, str):
+args_string = args
+elif platform.system() == 'Windows':
+# shlex not reliable on Windows so we use Use crude quoting with "...".
+args_string = ''
+for i, arg in enumerate(args):
+assert '"' not in arg
+if i:
+args_string += ' '
+args_string += f'"{arg}"'
+else:
+args_string = shlex.join(args)
+if platform.system() == 'Windows':
+command = f'{path}\\Scripts\\activate && python {args_string}'
+else:
+command = f'. {path}/bin/activate && python {args_string}'
+e = run(command, check=0)
+return e
 if __name__ == '__main__':
 # Internal-only limited command line support, used if
 # graal_legacy_python_config is true.
 #
 includes, ldflags = sysconfig_python_flags()
-if sys.argv[1:] == ['--graal-legacy-python-config', '--includes']:
+if sys.argv[1] == '--doctest':
+import doctest
+if sys.argv[2:]:
+for f in sys.argv[2:]:
+ff = globals()[f]
+doctest.run_docstring_examples(ff, globals())
+else:
+doctest.testmod(None)
+elif sys.argv[1:] == ['--graal-legacy-python-config', '--includes']:
 print(includes)
 elif sys.argv[1:] == ['--graal-legacy-python-config', '--ldflags']:
 print(ldflags)
 else:
 assert 0, f'Expected `--graal-legacy-python-config --includes|--ldflags` but {sys.argv=}'

Mercurial > hgrepos > Python2 > PyMuPDF

comparison pipcl.py @ 41:71bcc18e306f