Mercurial > hgrepos > Python2 > PyMuPDF

comparison setup.py @ 1:1d09e1dec1d9 upstream

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
ADD: PyMuPDF v1.26.4: the original sdist. It does not yet contain MuPDF. This normally will be downloaded when building PyMuPDF.
author Franz Glasner <fzglas.hg@dom66.de>
date Mon, 15 Sep 2025 11:37:51 +0200
parents
children 5ab937c03c27 a6bc019ac0b2
comparison
equal deleted inserted replaced
-1:000000000000 1:1d09e1dec1d9
1 #! /usr/bin/env python3
2
3 '''
4 Overview:
5
6 Build script for PyMuPDF, supporting PEP-517 and simple command-line usage.
7
8 We hard-code the URL of the MuPDF .tar.gz file that we require. This
9 generally points to a particular source release on mupdf.com.
10
11 Default behaviour:
12
13 Building an sdist:
14 As of 2024-002-28 we no longer download the MuPDF .tar.gz file and
15 embed it within the sdist. Instead it will be downloaded at build
16 time.
17
18 Building PyMuPDF:
19 We first download the hard-coded mupdf .tar.gz file.
20
21 Then we extract and build MuPDF locally, before building PyMuPDF
22 itself. So PyMuPDF will always be built with the exact MuPDF
23 release that we require.
24
25
26 Environmental variables:
27
28 If building with system MuPDF (PYMUPDF_SETUP_MUPDF_BUILD is empty string):
29
30 CFLAGS
31 CXXFLAGS
32 LDFLAGS
33 Added to c, c++, and link commands.
34
35 PYMUPDF_INCLUDES
36 Colon-separated extra include paths.
37
38 PYMUPDF_MUPDF_LIB
39 Directory containing MuPDF libraries, (libmupdf.so,
40 libmupdfcpp.so).
41
42 PYMUPDF_SETUP_DEVENV
43 Location of devenv.com on Windows. If unset we search for it - see
44 wdev.py. if that fails we use just 'devenv.com'.
45
46 PYMUPDF_SETUP_DUMMY
47 If 1, we build dummy sdist and wheel with no files.
48
49 PYMUPDF_SETUP_FLAVOUR
50 Control building of separate wheels for PyMuPDF.
51
52 Must be unset or a combination of 'p', 'b' and 'd'.
53
54 Default is 'pbd'.
55
56 'p':
57 Generated wheel contains PyMuPDF code.
58 'b':
59 Generated wheel contains MuPDF libraries; these are independent of
60 the Python version.
61 'd':
62 Generated wheel contains includes and libraries for MuPDF.
63
64 If 'p' is included, the generated wheel is called PyMuPDF.
65 Otherwise if 'b' is included the generated wheel is called PyMuPDFb.
66 Otherwise if 'd' is included the generated wheel is called PyMuPDFd.
67
68 For example:
69
70 'pb': a `PyMuPDF` wheel with PyMuPDF runtime files and MuPDF
71 runtime shared libraries.
72
73 'b': a `PyMuPDFb` wheel containing MuPDF runtime shared libraries.
74
75 'pbd' a `PyMuPDF` wheel with PyMuPDF runtime files and MuPDF
76 runtime shared libraries, plus MuPDF build-time files (includes,
77 *.lib files on Windows).
78
79 'd': a `PyMuPDFd` wheel containing MuPDF build-time files
80 (includes, *.lib files on Windows).
81
82 PYMUPDF_SETUP_LIBCLANG
83 For internal testing.
84
85 PYMUPDF_SETUP_MUPDF_BUILD
86 If unset or '-', use internal hard-coded default MuPDF location.
87 Otherwise overrides location of MuPDF when building PyMuPDF:
88 Empty string:
89 Build PyMuPDF with the system MuPDF.
90 A string starting with 'git:':
91 Use `git clone` to get a MuPDF checkout. We use the
92 string in the git clone command; it must contain the git
93 URL from which to clone, and can also contain other `git
94 clone` args, for example:
95 PYMUPDF_SETUP_MUPDF_BUILD="git:--branch master https://github.com/ArtifexSoftware/mupdf.git"
96 Otherwise:
97 Location of mupdf directory.
98
99 PYMUPDF_SETUP_MUPDF_BSYMBOLIC
100 If '0' we do not link libmupdf.so with -Bsymbolic.
101
102 PYMUPDF_SETUP_MUPDF_TESSERACT
103 If '0' we build MuPDF without Tesseract.
104
105 PYMUPDF_SETUP_MUPDF_BUILD_TYPE
106 Unix only. Controls build type of MuPDF. Supported values are:
107 debug
108 memento
109 release (default)
110
111 PYMUPDF_SETUP_MUPDF_CLEAN
112 Unix only. If '1', we do a clean MuPDF build.
113
114 PYMUPDF_SETUP_MUPDF_REFCHECK_IF
115 Should be preprocessor statement to enable MuPDF reference count
116 checking.
117
118 As of 2024-09-27, MuPDF default is `#ifndef NDEBUG`.
119
120 PYMUPDF_SETUP_MUPDF_TRACE_IF
121 Should be preprocessor statement to enable MuPDF runtime diagnostics in
122 response to environment variables such as MUPDF_trace.
123
124 As of 2024-09-27, MuPDF default is `#ifndef NDEBUG`.
125
126 PYMUPDF_SETUP_MUPDF_THIRD
127 If '0' and we are building on Linux with the system MuPDF
128 (i.e. PYMUPDF_SETUP_MUPDF_BUILD=''), then don't link with
129 `-lmupdf-third`.
130
131 PYMUPDF_SETUP_MUPDF_VS_UPGRADE
132 If '1' we run mupdf `scripts/mupdfwrap.py` with `--vs-upgrade 1` to
133 help Windows builds work with Visual Studio versions newer than 2019.
134
135 PYMUPDF_SETUP_MUPDF_TGZ
136 If set, overrides location of MuPDF .tar.gz file:
137 Empty string:
138 Do not download MuPDF .tar.gz file. Sdist's will not contain
139 MuPDF.
140
141 A string containing '://':
142 The URL from which to download the MuPDF .tar.gz file. Leaf
143 must match mupdf-*.tar.gz.
144
145 Otherwise:
146 The path of local mupdf git checkout. We put all files in this
147 checkout known to git into a local tar archive.
148
149 PYMUPDF_SETUP_MUPDF_OVERWRITE_CONFIG
150 If '0' we do not overwrite MuPDF's include/mupdf/fitz/config.h with
151 PyMuPDF's own configuration file, before building MuPDF.
152
153 PYMUPDF_SETUP_MUPDF_REBUILD
154 If 0 we do not (re)build mupdf.
155
156 PYMUPDF_SETUP_PY_LIMITED_API
157 If not '0', we build for current Python's stable ABI.
158
159 However if unset and we are on Python-3.13 or later, we do
160 not build for the stable ABI because as of 2025-03-04 SWIG
161 generates incorrect stable ABI code with Python-3.13 - see:
162 https://github.com/swig/swig/issues/3059
163
164 PYMUPDF_SETUP_URL_WHEEL
165 If set, we use an existing wheel instead of building a new wheel.
166
167 If starts with `http://` or `https://`:
168 If ends with '/', we append our wheel name and download. Otherwise
169 we download directly.
170
171 If starts with `file://`:
172 If ends with '/' we look for a matching wheel name, `using
173 pipcl.wheel_name_match()` to cope with differing platform tags,
174 for example our `manylinux2014_x86_64` will match with an existing
175 wheel with `manylinux2014_x86_64.manylinux_2_17_x86_64`.
176
177 Any other prefix is an error.
178
179 PYMUPDF_SETUP_SWIG
180 If set, we use this instead of `swig`.
181
182 WDEV_VS_YEAR
183 If set, we use as Visual Studio year, for example '2019' or '2022'.
184
185 WDEV_VS_GRADE
186 If set, we use as Visual Studio grade, for example 'Community' or
187 'Professional' or 'Enterprise'.
188 '''
189
190 import glob
191 import io
192 import os
193 import textwrap
194 import time
195 import platform
196 import re
197 import shlex
198 import shutil
199 import stat
200 import subprocess
201 import sys
202 import tarfile
203 import traceback
204 import urllib.request
205 import zipfile
206
207 import pipcl
208
209
210 log = pipcl.log0
211
212 run = pipcl.run
213
214
215 if 1:
216 # For debugging.
217 log(f'### Starting.')
218 pipcl.show_system()
219
220
221 PYMUPDF_SETUP_FLAVOUR = os.environ.get( 'PYMUPDF_SETUP_FLAVOUR', 'pbd')
222 for i in PYMUPDF_SETUP_FLAVOUR:
223 assert i in 'pbd', f'Unrecognised flag "{i} in {PYMUPDF_SETUP_FLAVOUR=}. Should be one of "p", "b", "d"'
224
225 g_root = os.path.abspath( f'{__file__}/..')
226
227 # Name of file that identifies that we are in a PyMuPDF sdist.
228 g_pymupdfb_sdist_marker = 'pymupdfb_sdist'
229
230 python_version_tuple = tuple(int(x) for x in platform.python_version_tuple()[:2])
231
232 PYMUPDF_SETUP_PY_LIMITED_API = os.environ.get('PYMUPDF_SETUP_PY_LIMITED_API')
233 assert PYMUPDF_SETUP_PY_LIMITED_API in (None, '', '0', '1'), \
234 f'Should be "", "0", "1" or undefined: {PYMUPDF_SETUP_PY_LIMITED_API=}.'
235 if PYMUPDF_SETUP_PY_LIMITED_API is None and python_version_tuple >= (3, 13):
236 log(f'Not defaulting to Python limited api because {platform.python_version_tuple()=}.')
237 g_py_limited_api = False
238 else:
239 g_py_limited_api = (PYMUPDF_SETUP_PY_LIMITED_API != '0')
240
241 PYMUPDF_SETUP_URL_WHEEL = os.environ.get('PYMUPDF_SETUP_URL_WHEEL')
242 log(f'{PYMUPDF_SETUP_URL_WHEEL=}')
243
244 PYMUPDF_SETUP_DUMMY = os.environ.get('PYMUPDF_SETUP_DUMMY')
245 log(f'{PYMUPDF_SETUP_DUMMY=}')
246
247 PYMUPDF_SETUP_SWIG = os.environ.get('PYMUPDF_SETUP_SWIG')
248
249 def _fs_remove(path):
250 '''
251 Removes file or directory, without raising exception if it doesn't exist.
252
253 We assert-fail if the path still exists when we return, in case of
254 permission problems etc.
255 '''
256 # First try deleting `path` as a file.
257 try:
258 os.remove( path)
259 except Exception as e:
260 pass
261
262 if os.path.exists(path):
263 # Try deleting `path` as a directory. Need to use
264 # shutil.rmtree() callback to handle permission problems; see:
265 # https://docs.python.org/3/library/shutil.html#rmtree-example
266 #
267 def error_fn(fn, path, excinfo):
268 # Clear the readonly bit and reattempt the removal.
269 os.chmod(path, stat.S_IWRITE)
270 fn(path)
271 shutil.rmtree( path, onerror=error_fn)
272
273 assert not os.path.exists( path)
274
275
276 def _git_get_branch( directory):
277 command = f'cd {directory} && git branch --show-current'
278 log( f'Running: {command}')
279 p = subprocess.run(
280 command,
281 shell=True,
282 check=False,
283 text=True,
284 stdout=subprocess.PIPE,
285 )
286 ret = None
287 if p.returncode == 0:
288 ret = p.stdout.strip()
289 log( f'Have found MuPDF git branch: ret={ret!r}')
290 return ret
291
292
293 def tar_check(path, mode='r:gz', prefix=None, remove=False):
294 '''
295 Checks items in tar file have same <top-directory>, or <prefix> if not None.
296
297 We fail if items in tar file have different top-level directory names.
298
299 path:
300 The tar file.
301 mode:
302 As tarfile.open().
303 prefix:
304 If not None, we fail if tar file's <top-directory> is not <prefix>.
305
306 Returns the directory name (which will be <prefix> if not None).
307 '''
308 with tarfile.open( path, mode) as t:
309 items = t.getnames()
310 assert items
311 item = items[0]
312 assert not item.startswith('./') and not item.startswith('../')
313 s = item.find('/')
314 if s == -1:
315 prefix_actual = item + '/'
316 else:
317 prefix_actual = item[:s+1]
318 if prefix:
319 assert prefix == prefix_actual, f'{path=} {prefix=} {prefix_actual=}'
320 for item in items[1:]:
321 assert item.startswith( prefix_actual), f'prefix_actual={prefix_actual!r} != item={item!r}'
322 return prefix_actual
323
324
325 def tar_extract(path, mode='r:gz', prefix=None, exists='raise'):
326 '''
327 Extracts tar file into single local directory.
328
329 We fail if items in tar file have different <top-directory>.
330
331 path:
332 The tar file.
333 mode:
334 As tarfile.open().
335 prefix:
336 If not None, we fail if tar file's <top-directory> is not <prefix>.
337 exists:
338 What to do if <top-directory> already exists:
339 'raise': raise exception.
340 'remove': remove existing file/directory before extracting.
341 'return': return without extracting.
342
343 Returns the directory name (which will be <prefix> if not None, with '/'
344 appended if not already present).
345 '''
346 prefix_actual = tar_check( path, mode, prefix)
347 if os.path.exists( prefix_actual):
348 if exists == 'raise':
349 raise Exception( f'Path already exists: {prefix_actual!r}')
350 elif exists == 'remove':
351 remove( prefix_actual)
352 elif exists == 'return':
353 log( f'Not extracting {path} because already exists: {prefix_actual}')
354 return prefix_actual
355 else:
356 assert 0, f'Unrecognised exists={exists!r}'
357 assert not os.path.exists( prefix_actual), f'Path already exists: {prefix_actual}'
358 log( f'Extracting {path}')
359 with tarfile.open( path, mode) as t:
360 t.extractall()
361 return prefix_actual
362
363
364 def git_info( directory):
365 '''
366 Returns `(sha, comment, diff, branch)`, all items are str or None if not
367 available.
368
369 directory:
370 Root of git checkout.
371 '''
372 sha, comment, diff, branch = '', '', '', ''
373 cp = subprocess.run(
374 f'cd {directory} && (PAGER= git show --pretty=oneline|head -n 1 && git diff)',
375 capture_output=1,
376 shell=1,
377 text=1,
378 )
379 if cp.returncode == 0:
380 sha, _ = cp.stdout.split(' ', 1)
381 comment, diff = _.split('\n', 1)
382 cp = subprocess.run(
383 f'cd {directory} && git rev-parse --abbrev-ref HEAD',
384 capture_output=1,
385 shell=1,
386 text=1,
387 )
388 if cp.returncode == 0:
389 branch = cp.stdout.strip()
390 log(f'git_info(): directory={directory!r} returning branch={branch!r} sha={sha!r} comment={comment!r}')
391 return sha, comment, diff, branch
392
393
394 def git_patch(directory, patch, hard=False):
395 '''
396 Applies string <patch> with `git patch` in <directory>.
397
398 If <hard> is true we clean the tree with `git checkout .` and then apply
399 the patch.
400
401 Otherwise we apply patch only if it is not already applied; this might fail
402 if there are conflicting changes in the tree.
403 '''
404 log(f'Applying patch in {directory}:\n{textwrap.indent(patch, " ")}')
405 if not patch:
406 return
407 # Carriage returns break `git apply` so we use `newline='\n'` in open().
408 path = os.path.abspath(f'{directory}/pymupdf_patch.txt')
409 with open(path, 'w', newline='\n') as f:
410 f.write(patch)
411 log(f'Using patch file: {path}')
412 if hard:
413 run(f'cd {directory} && git checkout .')
414 run(f'cd {directory} && git apply {path}')
415 log(f'Have applied patch in {directory}.')
416 else:
417 e = run( f'cd {directory} && git apply --check --reverse {path}', check=0)
418 if e == 0:
419 log(f'Not patching {directory} because already patched.')
420 else:
421 run(f'cd {directory} && git apply {path}')
422 log(f'Have applied patch in {directory}.')
423 run(f'cd {directory} && git diff')
424
425
426 mupdf_tgz = os.path.abspath( f'{__file__}/../mupdf.tgz')
427
428 def get_mupdf_internal(out, location=None, sha=None, local_tgz=None):
429 '''
430 Gets MuPDF as either a .tgz or a local directory.
431
432 Args:
433 out:
434 Either 'dir' (we return name of local directory containing mupdf) or 'tgz' (we return
435 name of local .tgz file containing mupdf).
436 location:
437 First, if None we set to hard-coded default URL or git location.
438 If starts with 'git:', should be remote git location.
439 Otherwise if containing '://' should be URL for .tgz.
440 Otherwise should path of local mupdf checkout.
441 sha:
442 If not None and we use git clone, we checkout this sha.
443 local_tgz:
444 If not None, must be local .tgz file.
445 Returns:
446 (path, location):
447 `path` is absolute path of local directory or .tgz containing
448 MuPDF, or None if we are to use system MuPDF.
449
450 `location_out` is `location` if not None, else the hard-coded
451 default location.
452
453 '''
454 log(f'get_mupdf_internal(): {out=} {location=} {sha=}')
455 assert out in ('dir', 'tgz')
456 if location is None:
457 location = f'https://mupdf.com/downloads/archive/mupdf-{version_mupdf}-source.tar.gz'
458 #location = 'git:--branch master https://github.com/ArtifexSoftware/mupdf.git'
459
460 if location == '':
461 # Use system mupdf.
462 return None, location
463
464 local_dir = None
465 if local_tgz:
466 assert os.path.isfile(local_tgz)
467 elif location.startswith( 'git:'):
468 location_git = location[4:]
469 local_dir = 'mupdf-git'
470
471 # Try to update existing checkout.
472 e = run(f'cd {local_dir} && git pull && git submodule update --init', check=False)
473 if e:
474 # No existing git checkout, so do a fresh clone.
475 _fs_remove(local_dir)
476 gitargs = location[4:]
477 run(f'git clone --recursive --depth 1 --shallow-submodules {gitargs} {local_dir}')
478
479 # Show sha of checkout.
480 run( f'cd {local_dir} && git show --pretty=oneline|head -n 1', check=False)
481 if sha:
482 run( f'cd {local_dir} && git checkout {sha}')
483 elif '://' in location:
484 # Download .tgz.
485 local_tgz = os.path.basename( location)
486 suffix = '.tar.gz'
487 assert location.endswith(suffix), f'Unrecognised suffix in remote URL {location=}.'
488 name = local_tgz[:-len(suffix)]
489 log( f'Download {location=} {local_tgz=} {name=}')
490 if os.path.exists(local_tgz):
491 try:
492 tar_check(local_tgz, 'r:gz', prefix=f'{name}/')
493 except Exception as e:
494 log(f'Not using existing file {local_tgz} because invalid tar data: {e}')
495 _fs_remove( local_tgz)
496 if os.path.exists(local_tgz):
497 log(f'Not downloading from {location} because already present: {local_tgz!r}')
498 else:
499 log(f'Downloading from {location=} to {local_tgz=}.')
500 urllib.request.urlretrieve( location, local_tgz + '-')
501 os.rename(local_tgz + '-', local_tgz)
502 assert os.path.exists( local_tgz)
503 tar_check( local_tgz, 'r:gz', prefix=f'{name}/')
504 else:
505 assert os.path.isdir(location), f'Local MuPDF does not exist: {location=}'
506 local_dir = location
507
508 assert bool(local_dir) != bool(local_tgz)
509 if out == 'dir':
510 if not local_dir:
511 assert local_tgz
512 local_dir = tar_extract( local_tgz, exists='return')
513 return os.path.abspath( local_dir), location
514 elif out == 'tgz':
515 if not local_tgz:
516 # Create .tgz containing git files in `local_dir`.
517 assert local_dir
518 if local_dir.endswith( '/'):
519 local_dir = local_dir[:-1]
520 top = os.path.basename(local_dir)
521 local_tgz = f'{local_dir}.tgz'
522 log( f'Creating .tgz from git files. {top=} {local_dir=} {local_tgz=}')
523 _fs_remove( local_tgz)
524 with tarfile.open( local_tgz, 'w:gz') as f:
525 for name in pipcl.git_items( local_dir, submodules=True):
526 path = os.path.join( local_dir, name)
527 if os.path.isfile( path):
528 path2 = f'{top}/{name}'
529 log(f'Adding {path=} {path2=}.')
530 f.add( path, path2, recursive=False)
531 return os.path.abspath( local_tgz), location
532 else:
533 assert 0, f'Unrecognised {out=}'
534
535
536
537 def get_mupdf_tgz():
538 '''
539 Creates .tgz file called containing MuPDF source, for inclusion in an
540 sdist.
541
542 What we do depends on environmental variable PYMUPDF_SETUP_MUPDF_TGZ; see
543 docs at start of this file for details.
544
545 Returns name of top-level directory within the .tgz file.
546 '''
547 name, location = get_mupdf_internal( 'tgz', os.environ.get('PYMUPDF_SETUP_MUPDF_TGZ'))
548 return name, location
549
550
551 def get_mupdf(path=None, sha=None):
552 '''
553 Downloads and/or extracts mupdf and returns (path, location) where `path`
554 is the local mupdf directory and `location` is where it came from.
555
556 Exact behaviour depends on environmental variable
557 PYMUPDF_SETUP_MUPDF_BUILD; see docs at start of this file for details.
558 '''
559 m = os.environ.get('PYMUPDF_SETUP_MUPDF_BUILD')
560 if m == '-':
561 # This allows easy specification in Github actions.
562 m = None
563 if m is None and os.path.isfile(mupdf_tgz):
564 # This makes us use tgz inside sdist.
565 log(f'Using local tgz: {mupdf_tgz=}')
566 return get_mupdf_internal('dir', local_tgz=mupdf_tgz)
567 return get_mupdf_internal('dir', m)
568
569
570 linux = sys.platform.startswith( 'linux') or 'gnu' in sys.platform
571 openbsd = sys.platform.startswith( 'openbsd')
572 freebsd = sys.platform.startswith( 'freebsd')
573 darwin = sys.platform.startswith( 'darwin')
574 windows = platform.system() == 'Windows' or platform.system().startswith('CYGWIN')
575 msys2 = platform.system().startswith('MSYS_NT-')
576
577 pyodide_flags = '-fwasm-exceptions'
578
579 if os.environ.get('PYODIDE') == '1':
580 if os.environ.get('OS') != 'pyodide':
581 log('PYODIDE=1, setting OS=pyodide.')
582 os.environ['OS'] = 'pyodide'
583 os.environ['XCFLAGS'] = pyodide_flags
584 os.environ['XCXXFLAGS'] = pyodide_flags
585
586 pyodide = os.environ.get('OS') == 'pyodide'
587
588 def build():
589 '''
590 pipcl.py `build_fn()` callback.
591 '''
592 #pipcl.show_sysconfig()
593
594 if PYMUPDF_SETUP_DUMMY == '1':
595 log(f'{PYMUPDF_SETUP_DUMMY=} Building dummy wheel with no files.')
596 return list()
597
598 # Download MuPDF.
599 #
600 mupdf_local, mupdf_location = get_mupdf()
601 if mupdf_local:
602 mupdf_version_tuple = get_mupdf_version(mupdf_local)
603 # else we cannot determine version this way and do not use it
604
605 build_type = os.environ.get( 'PYMUPDF_SETUP_MUPDF_BUILD_TYPE', 'release')
606 assert build_type in ('debug', 'memento', 'release'), \
607 f'Unrecognised build_type={build_type!r}'
608
609 overwrite_config = os.environ.get('PYMUPDF_SETUP_MUPDF_OVERWRITE_CONFIG', '1') == '1'
610
611 PYMUPDF_SETUP_MUPDF_REFCHECK_IF = os.environ.get('PYMUPDF_SETUP_MUPDF_REFCHECK_IF')
612 PYMUPDF_SETUP_MUPDF_TRACE_IF = os.environ.get('PYMUPDF_SETUP_MUPDF_TRACE_IF')
613
614 # Build MuPDF shared libraries.
615 #
616 if windows:
617 mupdf_build_dir = build_mupdf_windows(
618 mupdf_local,
619 build_type,
620 overwrite_config,
621 g_py_limited_api,
622 PYMUPDF_SETUP_MUPDF_REFCHECK_IF,
623 PYMUPDF_SETUP_MUPDF_TRACE_IF,
624 )
625 else:
626 if 'p' not in PYMUPDF_SETUP_FLAVOUR and 'b' not in PYMUPDF_SETUP_FLAVOUR:
627 # We only need MuPDF headers, so no point building MuPDF.
628 log(f'Not building MuPDF because not Windows and {PYMUPDF_SETUP_FLAVOUR=}.')
629 mupdf_build_dir = None
630 else:
631 mupdf_build_dir = build_mupdf_unix(
632 mupdf_local,
633 build_type,
634 overwrite_config,
635 g_py_limited_api,
636 PYMUPDF_SETUP_MUPDF_REFCHECK_IF,
637 PYMUPDF_SETUP_MUPDF_TRACE_IF,
638 PYMUPDF_SETUP_SWIG,
639 )
640 log( f'build(): mupdf_build_dir={mupdf_build_dir!r}')
641
642 # Build rebased `extra` module.
643 #
644 if 'p' in PYMUPDF_SETUP_FLAVOUR:
645 path_so_leaf = _build_extension(
646 mupdf_local,
647 mupdf_build_dir,
648 build_type,
649 g_py_limited_api,
650 )
651 else:
652 log(f'Not building extension.')
653 path_so_leaf = None
654
655 # Generate list of (from, to) items to return to pipcl. What we add depends
656 # on PYMUPDF_SETUP_FLAVOUR.
657 #
658 ret = list()
659 def add(flavour, from_, to_):
660 assert flavour in 'pbd'
661 if flavour in PYMUPDF_SETUP_FLAVOUR:
662 ret.append((from_, to_))
663
664 to_dir = 'pymupdf/'
665 to_dir_d = f'{to_dir}/mupdf-devel'
666
667 # Add implementation files.
668 add('p', f'{g_root}/src/__init__.py', to_dir)
669 add('p', f'{g_root}/src/__main__.py', to_dir)
670 add('p', f'{g_root}/src/pymupdf.py', to_dir)
671 add('p', f'{g_root}/src/table.py', to_dir)
672 add('p', f'{g_root}/src/utils.py', to_dir)
673 add('p', f'{g_root}/src/_wxcolors.py', to_dir)
674 add('p', f'{g_root}/src/_apply_pages.py', to_dir)
675 add('p', f'{g_root}/src/build/extra.py', to_dir)
676 if path_so_leaf:
677 add('p', f'{g_root}/src/build/{path_so_leaf}', to_dir)
678
679 # Add support for `fitz` backwards compatibility.
680 add('p', f'{g_root}/src/fitz___init__.py', 'fitz/__init__.py')
681 add('p', f'{g_root}/src/fitz_table.py', 'fitz/table.py')
682 add('p', f'{g_root}/src/fitz_utils.py', 'fitz/utils.py')
683
684 if mupdf_local:
685 # Add MuPDF Python API.
686 add('p', f'{mupdf_build_dir}/mupdf.py', to_dir)
687
688 # Add MuPDF shared libraries.
689 if windows:
690 wp = pipcl.wdev.WindowsPython()
691 add('p', f'{mupdf_build_dir}/_mupdf.pyd', to_dir)
692 add('b', f'{mupdf_build_dir}/mupdfcpp{wp.cpu.windows_suffix}.dll', to_dir)
693
694 # Add Windows .lib files.
695 mupdf_build_dir2 = _windows_lib_directory(mupdf_local, build_type)
696 add('d', f'{mupdf_build_dir2}/mupdfcpp{wp.cpu.windows_suffix}.lib', f'{to_dir_d}/lib/')
697 if mupdf_version_tuple >= (1, 26):
698 # MuPDF-1.25+ language bindings build also builds libmuthreads.
699 add('d', f'{mupdf_build_dir2}/libmuthreads.lib', f'{to_dir_d}/lib/')
700 elif darwin:
701 add('p', f'{mupdf_build_dir}/_mupdf.so', to_dir)
702 add('b', f'{mupdf_build_dir}/libmupdfcpp.so', to_dir)
703 add('b', f'{mupdf_build_dir}/libmupdf.dylib', to_dir)
704 add('d', f'{mupdf_build_dir}/libmupdf-threads.a', f'{to_dir_d}/lib/')
705 elif pyodide:
706 add('p', f'{mupdf_build_dir}/_mupdf.so', to_dir)
707 add('b', f'{mupdf_build_dir}/libmupdfcpp.so', 'PyMuPDF.libs/')
708 add('b', f'{mupdf_build_dir}/libmupdf.so', 'PyMuPDF.libs/')
709 else:
710 add('p', f'{mupdf_build_dir}/_mupdf.so', to_dir)
711 add('b', pipcl.get_soname(f'{mupdf_build_dir}/libmupdfcpp.so'), to_dir)
712 add('b', pipcl.get_soname(f'{mupdf_build_dir}/libmupdf.so'), to_dir)
713 add('d', f'{mupdf_build_dir}/libmupdf-threads.a', f'{to_dir_d}/lib/')
714
715 if 'd' in PYMUPDF_SETUP_FLAVOUR:
716 # Add MuPDF C and C++ headers to `ret_d`. Would prefer to use
717 # pipcl.git_items() but hard-coded mupdf tree is not a git
718 # checkout.
719 #
720 for root in (
721 f'{mupdf_local}/include',
722 f'{mupdf_local}/platform/c++/include',
723 ):
724 for dirpath, dirnames, filenames in os.walk(root):
725 for filename in filenames:
726 if not filename.endswith('.h'):
727 continue
728 header_abs = os.path.join(dirpath, filename)
729 assert header_abs.startswith(root)
730 header_rel = header_abs[len(root)+1:]
731 add('d', f'{header_abs}', f'{to_dir_d}/include/{header_rel}')
732
733 # Add a .py file containing location of MuPDF.
734 try:
735 sha, comment, diff, branch = git_info(g_root)
736 except Exception as e:
737 log(f'Failed to get git information: {e}')
738 sha, comment, diff, branch = (None, None, None, None)
739 swig = PYMUPDF_SETUP_SWIG or 'swig'
740 swig_version_text = run(f'{swig} --version', capture=1)
741 m = re.search('\nSWIG Version ([^\n]+)', swig_version_text)
742 log(f'{swig_version_text=}')
743 assert m, f'Unrecognised {swig_version_text=}'
744 swig_version = m.group(1)
745 def int_or_0(text):
746 try:
747 return int(text)
748 except Exception:
749 return 0
750 swig_version_tuple = tuple(int_or_0(i) for i in swig_version.split('.'))
751 log(f'{swig_version=}')
752 text = ''
753 text += f'mupdf_location = {mupdf_location!r}\n'
754 text += f'pymupdf_version = {version_p!r}\n'
755 text += f'pymupdf_git_sha = {sha!r}\n'
756 text += f'pymupdf_git_diff = {diff!r}\n'
757 text += f'pymupdf_git_branch = {branch!r}\n'
758 text += f'swig_version = {swig_version!r}\n'
759 text += f'swig_version_tuple = {swig_version_tuple!r}\n'
760 add('p', text.encode(), f'{to_dir}/_build.py')
761
762 # Add single README file.
763 if 'p' in PYMUPDF_SETUP_FLAVOUR:
764 add('p', f'{g_root}/README.md', '$dist-info/README.md')
765 elif 'b' in PYMUPDF_SETUP_FLAVOUR:
766 add('b', f'{g_root}/READMEb.md', '$dist-info/README.md')
767 elif 'd' in PYMUPDF_SETUP_FLAVOUR:
768 add('d', f'{g_root}/READMEd.md', '$dist-info/README.md')
769
770 return ret
771
772
773 def env_add(env, name, value, sep=' ', prepend=False, verbose=False):
774 '''
775 Appends/prepends `<value>` to `env[name]`.
776
777 If `name` is not in `env`, we use os.environ[name] if it exists.
778 '''
779 v = env.get(name)
780 if verbose:
781 log(f'Initally: {name}={v!r}')
782 if v is None:
783 v = os.environ.get(name)
784 if v is None:
785 env[ name] = value
786 else:
787 if prepend:
788 env[ name] = f'{value}{sep}{v}'
789 else:
790 env[ name] = f'{v}{sep}{value}'
791 if verbose:
792 log(f'Returning with {name}={env[name]!r}')
793
794
795 def build_mupdf_windows(
796 mupdf_local,
797 build_type,
798 overwrite_config,
799 g_py_limited_api,
800 PYMUPDF_SETUP_MUPDF_REFCHECK_IF,
801 PYMUPDF_SETUP_MUPDF_TRACE_IF,
802 ):
803
804 assert mupdf_local
805
806 if overwrite_config:
807 mupdf_config_h = f'{mupdf_local}/include/mupdf/fitz/config.h'
808 prefix = '#define TOFU_CJK_EXT 1 /* PyMuPDF override. */\n'
809 with open(mupdf_config_h) as f:
810 text = f.read()
811 if text.startswith(prefix):
812 print(f'Not modifying {mupdf_config_h} because already has prefix {prefix!r}.')
813 else:
814 print(f'Prefixing {mupdf_config_h} with {prefix!r}.')
815 text = prefix + text
816 st = os.stat(mupdf_config_h)
817 with open(mupdf_config_h, 'w') as f:
818 f.write(text)
819 os.utime(mupdf_config_h, (st.st_atime, st.st_mtime))
820
821 wp = pipcl.wdev.WindowsPython()
822 tesseract = '' if os.environ.get('PYMUPDF_SETUP_MUPDF_TESSERACT') == '0' else 'tesseract-'
823 windows_build_tail = f'build\\shared-{tesseract}{build_type}'
824 if g_py_limited_api:
825 windows_build_tail += f'-Py_LIMITED_API_{pipcl.current_py_limited_api()}'
826 windows_build_tail += f'-x{wp.cpu.bits}-py{wp.version}'
827 windows_build_dir = f'{mupdf_local}\\{windows_build_tail}'
828 #log( f'Building mupdf.')
829 devenv = os.environ.get('PYMUPDF_SETUP_DEVENV')
830 if not devenv:
831 try:
832 # Prefer VS-2022 as that is what Github provide in windows-2022.
833 log(f'Looking for Visual Studio 2022.')
834 vs = pipcl.wdev.WindowsVS(year=2022)
835 except Exception as e:
836 log(f'Failed to find VS-2022:\n'
837 f'{textwrap.indent(traceback.format_exc(), " ")}'
838 )
839 log(f'Looking for any Visual Studio.')
840 vs = pipcl.wdev.WindowsVS()
841 log(f'vs:\n{vs.description_ml(" ")}')
842 devenv = vs.devenv
843 if not devenv:
844 devenv = 'devenv.com'
845 log( f'Cannot find devenv.com in default locations, using: {devenv!r}')
846 command = f'cd "{mupdf_local}" && "{sys.executable}" ./scripts/mupdfwrap.py'
847 if os.environ.get('PYMUPDF_SETUP_MUPDF_VS_UPGRADE') == '1':
848 command += ' --vs-upgrade 1'
849
850 # Would like to simply do f'... --devenv {shutil.quote(devenv)}', but
851 # it looks like if `devenv` has spaces then `shutil.quote()` puts it
852 # inside single quotes, which then appear to be ignored when run by
853 # subprocess.run().
854 #
855 # So instead we strip any enclosing quotes and the enclose with
856 # double-quotes.
857 #
858 if len(devenv) >= 2:
859 for q in '"', "'":
860 if devenv.startswith( q) and devenv.endswith( q):
861 devenv = devenv[1:-1]
862 command += f' -d {windows_build_tail}'
863 command += f' -b'
864 if PYMUPDF_SETUP_MUPDF_REFCHECK_IF:
865 command += f' --refcheck-if "{PYMUPDF_SETUP_MUPDF_REFCHECK_IF}"'
866 if PYMUPDF_SETUP_MUPDF_TRACE_IF:
867 command += f' --trace-if "{PYMUPDF_SETUP_MUPDF_TRACE_IF}"'
868 command += f' --devenv "{devenv}"'
869 command += f' all'
870 if os.environ.get( 'PYMUPDF_SETUP_MUPDF_REBUILD') == '0':
871 log( f'PYMUPDF_SETUP_MUPDF_REBUILD is "0" so not building MuPDF; would have run: {command}')
872 else:
873 log( f'Building MuPDF by running: {command}')
874 subprocess.run( command, shell=True, check=True)
875 log( f'Finished building mupdf.')
876
877 return windows_build_dir
878
879
880 def _windows_lib_directory(mupdf_local, build_type):
881 ret = f'{mupdf_local}/platform/win32/'
882 if _cpu_bits() == 64:
883 ret += 'x64/'
884 if build_type == 'release':
885 ret += 'Release/'
886 elif build_type == 'debug':
887 ret += 'Debug/'
888 else:
889 assert 0, f'Unrecognised {build_type=}.'
890 return ret
891
892
893 def _cpu_bits():
894 if sys.maxsize == 2**31 - 1:
895 return 32
896 return 64
897
898
899 def build_mupdf_unix(
900 mupdf_local,
901 build_type,
902 overwrite_config,
903 g_py_limited_api,
904 PYMUPDF_SETUP_MUPDF_REFCHECK_IF,
905 PYMUPDF_SETUP_MUPDF_TRACE_IF,
906 PYMUPDF_SETUP_SWIG,
907 ):
908 '''
909 Builds MuPDF.
910
911 Args:
912 mupdf_local:
913 Path of MuPDF directory or None if we are using system MuPDF.
914
915 Returns the absolute path of build directory within MuPDF, e.g.
916 `.../mupdf/build/pymupdf-shared-release`, or `None` if we are using the
917 system MuPDF.
918 '''
919 if not mupdf_local:
920 log( f'Using system mupdf.')
921 return None
922
923 env = dict()
924 if overwrite_config:
925 # By predefining TOFU_CJK_EXT here, we don't need to modify
926 # MuPDF's include/mupdf/fitz/config.h.
927 log( f'Setting XCFLAGS and XCXXFLAGS to predefine TOFU_CJK_EXT.')
928 env_add(env, 'XCFLAGS', '-DTOFU_CJK_EXT')
929 env_add(env, 'XCXXFLAGS', '-DTOFU_CJK_EXT')
930
931 if openbsd or freebsd:
932 env_add(env, 'CXX', 'c++', ' ')
933
934 if darwin and os.environ.get('GITHUB_ACTIONS') == 'true':
935 if os.environ.get('ImageOS') == 'macos13':
936 # On Github macos13 we need to use Clang/LLVM (Homebrew) 15.0.7,
937 # otherwise mupdf:thirdparty/tesseract/src/api/baseapi.cpp fails to
938 # compile with:
939 #
940 # thirdparty/tesseract/src/api/baseapi.cpp:150:25: error: 'recursive_directory_iterator' is unavailable: introduced in macOS 10.15
941 #
942 # See:
943 # https://github.com/actions/runner-images/blob/main/images/macos/macos-13-Readme.md
944 #
945 log(f'Using llvm@15 clang and clang++')
946 cl15 = pipcl.run(f'brew --prefix llvm@15', capture=1)
947 log(f'{cl15=}')
948 cl15 = cl15.strip()
949 pipcl.run(f'ls -lL {cl15}')
950 pipcl.run(f'ls -lL {cl15}/bin')
951 cc = f'{cl15}/bin/clang'
952 cxx = f'{cl15}/bin/clang++'
953 env['CC'] = cc
954 env['CXX'] = cxx
955
956 # Show compiler versions.
957 cc = env.get('CC', 'cc')
958 cxx = env.get('CXX', 'c++')
959 pipcl.run(f'{cc} --version')
960 pipcl.run(f'{cxx} --version')
961
962 # Add extra flags for MacOS cross-compilation, where ARCHFLAGS can be
963 # '-arch arm64'.
964 #
965 archflags = os.environ.get( 'ARCHFLAGS')
966 if archflags:
967 env_add(env, 'XCFLAGS', archflags)
968 env_add(env, 'XLIBS', archflags)
969
970 mupdf_version_tuple = get_mupdf_version(mupdf_local)
971
972 # We specify a build directory path containing 'pymupdf' so that we
973 # coexist with non-PyMuPDF builds (because PyMuPDF builds have a
974 # different config.h).
975 #
976 # We also append further text to try to allow different builds to
977 # work if they reuse the mupdf directory.
978 #
979 # Using platform.machine() (e.g. 'amd64') ensures that different
980 # builds of mupdf on a shared filesystem can coexist. Using
981 # $_PYTHON_HOST_PLATFORM allows cross-compiled cibuildwheel builds
982 # to coexist, e.g. on github.
983 #
984 # Have experimented with looking at getconf_ARG_MAX to decide whether to
985 # omit `PyMuPDF-` from the build directory, to avoid command-too-long
986 # errors with mupdf-1.26. But it seems that `getconf ARG_MAX` returns
987 # a system limit, not the actual limit of the current shell, and there
988 # doesn't seem to be a way to find the current shell's limit.
989 #
990 build_prefix = f'PyMuPDF-'
991 if mupdf_version_tuple >= (1, 26):
992 # Avoid link command length problems seen on musllinux.
993 build_prefix = ''
994 if pyodide:
995 build_prefix += 'pyodide-'
996 else:
997 build_prefix += f'{platform.machine()}-'
998 build_prefix_extra = os.environ.get( '_PYTHON_HOST_PLATFORM')
999 if build_prefix_extra:
1000 build_prefix += f'{build_prefix_extra}-'
1001 build_prefix += 'shared-'
1002 if msys2:
1003 # Error in mupdf/scripts/tesseract/endianness.h:
1004 # #error "I don't know what architecture this is!"
1005 log(f'msys2: building MuPDF without tesseract.')
1006 elif os.environ.get('PYMUPDF_SETUP_MUPDF_TESSERACT') == '0':
1007 log(f'PYMUPDF_SETUP_MUPDF_TESSERACT=0 so building mupdf without tesseract.')
1008 else:
1009 build_prefix += 'tesseract-'
1010 if (
1011 linux
1012 and os.environ.get('PYMUPDF_SETUP_MUPDF_BSYMBOLIC', '1') == '1'
1013 ):
1014 log(f'Appending `bsymbolic-` to MuPDF build path.')
1015 build_prefix += 'bsymbolic-'
1016 log(f'{g_py_limited_api=}')
1017 if g_py_limited_api:
1018 build_prefix += f'Py_LIMITED_API_{pipcl.current_py_limited_api()}-'
1019 unix_build_dir = f'{mupdf_local}/build/{build_prefix}{build_type}'
1020 PYMUPDF_SETUP_MUPDF_CLEAN = os.environ.get('PYMUPDF_SETUP_MUPDF_CLEAN')
1021 if PYMUPDF_SETUP_MUPDF_CLEAN == '1':
1022 log(f'{PYMUPDF_SETUP_MUPDF_CLEAN=}, deleting {unix_build_dir=}.')
1023 shutil.rmtree(unix_build_dir, ignore_errors=1)
1024 # We need MuPDF's Python bindings, so we build MuPDF with
1025 # `mupdf/scripts/mupdfwrap.py` instead of running `make`.
1026 #
1027 command = f'cd {mupdf_local} &&'
1028 for n, v in env.items():
1029 command += f' {n}={shlex.quote(v)}'
1030 command += f' {sys.executable} ./scripts/mupdfwrap.py'
1031 if PYMUPDF_SETUP_SWIG:
1032 command += f' --swig {shlex.quote(PYMUPDF_SETUP_SWIG)}'
1033 command += f' -d build/{build_prefix}{build_type} -b'
1034 #command += f' --m-target libs'
1035 if PYMUPDF_SETUP_MUPDF_REFCHECK_IF:
1036 command += f' --refcheck-if "{PYMUPDF_SETUP_MUPDF_REFCHECK_IF}"'
1037 if PYMUPDF_SETUP_MUPDF_TRACE_IF:
1038 command += f' --trace-if "{PYMUPDF_SETUP_MUPDF_TRACE_IF}"'
1039 if 'p' in PYMUPDF_SETUP_FLAVOUR:
1040 command += ' all'
1041 else:
1042 command += ' m01' # No need for C++/Python bindings.
1043 command += f' && echo {unix_build_dir}:'
1044 command += f' && ls -l {unix_build_dir}'
1045
1046 if os.environ.get( 'PYMUPDF_SETUP_MUPDF_REBUILD') == '0':
1047 log( f'PYMUPDF_SETUP_MUPDF_REBUILD is "0" so not building MuPDF; would have run: {command}')
1048 else:
1049 log( f'Building MuPDF by running: {command}')
1050 subprocess.run( command, shell=True, check=True)
1051 log( f'Finished building mupdf.')
1052
1053 return unix_build_dir
1054
1055
1056 def get_mupdf_version(mupdf_dir):
1057 path = f'{mupdf_dir}/include/mupdf/fitz/version.h'
1058 with open(path) as f:
1059 text = f.read()
1060 v0 = re.search('#define FZ_VERSION_MAJOR ([0-9]+)', text)
1061 v1 = re.search('#define FZ_VERSION_MINOR ([0-9]+)', text)
1062 v2 = re.search('#define FZ_VERSION_PATCH ([0-9]+)', text)
1063 assert v0 and v1 and v2, f'Cannot find MuPDF version numbers in {path=}.'
1064 v0 = int(v0.group(1))
1065 v1 = int(v1.group(1))
1066 v2 = int(v2.group(1))
1067 return v0, v1, v2
1068
1069 def _fs_update(text, path):
1070 try:
1071 with open( path) as f:
1072 text0 = f.read()
1073 except OSError:
1074 text0 = None
1075 print(f'path={path!r} text==text0={text==text0!r}')
1076 if text != text0:
1077 with open( path, 'w') as f:
1078 f.write( text)
1079
1080
1081 def _build_extension( mupdf_local, mupdf_build_dir, build_type, g_py_limited_api):
1082 '''
1083 Builds Python extension module `_extra`.
1084
1085 Returns leafname of the generated shared libraries within mupdf_build_dir.
1086 '''
1087 (compiler_extra, linker_extra, includes, defines, optimise, debug, libpaths, libs, libraries) \
1088 = _extension_flags( mupdf_local, mupdf_build_dir, build_type)
1089 log(f'_build_extension(): {g_py_limited_api=} {defines=}')
1090 if mupdf_local:
1091 includes = (
1092 f'{mupdf_local}/platform/c++/include',
1093 f'{mupdf_local}/include',
1094 )
1095
1096 # Build rebased extension module.
1097 log('Building PyMuPDF rebased.')
1098 compile_extra_cpp = ''
1099 if darwin:
1100 # Avoids `error: cannot pass object of non-POD type
1101 # 'std::nullptr_t' through variadic function; call will abort at
1102 # runtime` when compiling `mupdf::pdf_dict_getl(..., nullptr)`.
1103 compile_extra_cpp += ' -Wno-non-pod-varargs'
1104 # Avoid errors caused by mupdf's C++ bindings' exception classes
1105 # not having `nothrow` to match the base exception class.
1106 compile_extra_cpp += ' -std=c++14'
1107 if windows:
1108 wp = pipcl.wdev.WindowsPython()
1109 libs = f'mupdfcpp{wp.cpu.windows_suffix}.lib'
1110 else:
1111 libs = ('mupdf', 'mupdfcpp')
1112 libraries = [
1113 f'{mupdf_build_dir}/libmupdf.so'
1114 f'{mupdf_build_dir}/libmupdfcpp.so'
1115 ]
1116
1117 path_so_leaf = pipcl.build_extension(
1118 name = 'extra',
1119 path_i = f'{g_root}/src/extra.i',
1120 outdir = f'{g_root}/src/build',
1121 includes = includes,
1122 defines = defines,
1123 libpaths = libpaths,
1124 libs = libs,
1125 compiler_extra = compiler_extra + compile_extra_cpp,
1126 linker_extra = linker_extra,
1127 optimise = optimise,
1128 debug = debug,
1129 prerequisites_swig = None,
1130 prerequisites_compile = f'{mupdf_local}/include',
1131 prerequisites_link = libraries,
1132 py_limited_api = g_py_limited_api,
1133 swig = PYMUPDF_SETUP_SWIG,
1134 )
1135
1136 return path_so_leaf
1137
1138
1139 def _extension_flags( mupdf_local, mupdf_build_dir, build_type):
1140 '''
1141 Returns various flags to pass to pipcl.build_extension().
1142 '''
1143 compiler_extra = ''
1144 linker_extra = ''
1145 if build_type == 'memento':
1146 compiler_extra += ' -DMEMENTO'
1147 if mupdf_build_dir:
1148 mupdf_build_dir_flags = os.path.basename( mupdf_build_dir).split( '-')
1149 else:
1150 mupdf_build_dir_flags = [build_type]
1151 optimise = 'release' in mupdf_build_dir_flags
1152 debug = 'debug' in mupdf_build_dir_flags
1153 r_extra = ''
1154 defines = list()
1155 if windows:
1156 defines.append('FZ_DLL_CLIENT')
1157 wp = pipcl.wdev.WindowsPython()
1158 if os.environ.get('PYMUPDF_SETUP_MUPDF_VS_UPGRADE') == '1':
1159 # MuPDF C++ build uses a parallel build tree with updated VS files.
1160 infix = 'win32-vs-upgrade'
1161 else:
1162 infix = 'win32'
1163 build_type_infix = 'Debug' if debug else 'Release'
1164 libpaths = (
1165 f'{mupdf_local}\\platform\\{infix}\\{wp.cpu.windows_subdir}{build_type_infix}',
1166 f'{mupdf_local}\\platform\\{infix}\\{wp.cpu.windows_subdir}{build_type_infix}Tesseract',
1167 )
1168 libs = f'mupdfcpp{wp.cpu.windows_suffix}.lib'
1169 libraries = f'{mupdf_local}\\platform\\{infix}\\{wp.cpu.windows_subdir}{build_type_infix}\\{libs}'
1170 compiler_extra = ''
1171 else:
1172 libs = ['mupdf']
1173 compiler_extra += (
1174 ' -Wall'
1175 ' -Wno-deprecated-declarations'
1176 ' -Wno-unused-const-variable'
1177 )
1178 if mupdf_local:
1179 libpaths = (mupdf_build_dir,)
1180 libraries = f'{mupdf_build_dir}/{libs[0]}'
1181 if openbsd:
1182 compiler_extra += ' -Wno-deprecated-declarations'
1183 else:
1184 libpaths = os.environ.get('PYMUPDF_MUPDF_LIB')
1185 libraries = None
1186 if libpaths:
1187 libpaths = libpaths.split(':')
1188
1189 if mupdf_local:
1190 includes = (
1191 f'{mupdf_local}/include',
1192 f'{mupdf_local}/include/mupdf',
1193 f'{mupdf_local}/thirdparty/freetype/include',
1194 )
1195 else:
1196 # Use system MuPDF.
1197 includes = list()
1198 pi = os.environ.get('PYMUPDF_INCLUDES')
1199 if pi:
1200 includes += pi.split(':')
1201 pmi = os.environ.get('PYMUPDF_MUPDF_INCLUDE')
1202 if pmi:
1203 includes.append(pmi)
1204 ldflags = os.environ.get('LDFLAGS')
1205 if ldflags:
1206 linker_extra += f' {ldflags}'
1207 cflags = os.environ.get('CFLAGS')
1208 if cflags:
1209 compiler_extra += f' {cflags}'
1210 cxxflags = os.environ.get('CXXFLAGS')
1211 if cxxflags:
1212 compiler_extra += f' {cxxflags}'
1213
1214 if pyodide:
1215 compiler_extra += f' {pyodide_flags}'
1216 linker_extra += f' {pyodide_flags}'
1217
1218 return compiler_extra, linker_extra, includes, defines, optimise, debug, libpaths, libs, libraries,
1219
1220
1221 def sdist():
1222 ret = list()
1223 if PYMUPDF_SETUP_DUMMY == '1':
1224 return ret
1225
1226 if PYMUPDF_SETUP_FLAVOUR == 'b':
1227 # Create a minimal sdist that will build/install a dummy PyMuPDFb.
1228 for p in (
1229 'setup.py',
1230 'pipcl.py',
1231 'wdev.py',
1232 'pyproject.toml',
1233 ):
1234 ret.append(p)
1235 ret.append(
1236 (
1237 b'This file indicates that we are a PyMuPDFb sdist and should build/install a dummy PyMuPDFb package.\n',
1238 g_pymupdfb_sdist_marker,
1239 )
1240 )
1241 return ret
1242
1243 for p in pipcl.git_items( g_root):
1244 if p.startswith(
1245 (
1246 'docs/',
1247 'signatures/',
1248 '.',
1249 )
1250 ):
1251 pass
1252 else:
1253 ret.append(p)
1254 if 0:
1255 tgz, mupdf_location = get_mupdf_tgz()
1256 if tgz:
1257 ret.append((tgz, mupdf_tgz))
1258 else:
1259 log(f'Not including MuPDF .tgz in sdist.')
1260 return ret
1261
1262
1263 classifier = [
1264 'Development Status :: 5 - Production/Stable',
1265 'Intended Audience :: Developers',
1266 'Intended Audience :: Information Technology',
1267 'Operating System :: MacOS',
1268 'Operating System :: Microsoft :: Windows',
1269 'Operating System :: POSIX :: Linux',
1270 'Programming Language :: C',
1271 'Programming Language :: C++',
1272 'Programming Language :: Python :: 3 :: Only',
1273 'Programming Language :: Python :: Implementation :: CPython',
1274 'Topic :: Utilities',
1275 'Topic :: Multimedia :: Graphics',
1276 'Topic :: Software Development :: Libraries',
1277 ]
1278
1279 # We generate different wheels depending on PYMUPDF_SETUP_FLAVOUR.
1280 #
1281
1282 # PyMuPDF version.
1283 version_p = '1.26.4'
1284
1285 version_mupdf = '1.26.7'
1286
1287 # PyMuPDFb version. This is the PyMuPDF version whose PyMuPDFb wheels we will
1288 # (re)use if generating separate PyMuPDFb wheels. Though as of PyMuPDF-1.24.11
1289 # (2024-10-03) we no longer use PyMuPDFb wheels so this is actually unused.
1290 #
1291 version_b = '1.26.3'
1292
1293 if os.path.exists(f'{g_root}/{g_pymupdfb_sdist_marker}'):
1294
1295 # We are in a PyMuPDFb sdist. We specify a dummy package so that pip builds
1296 # from sdists work - pip's build using PyMuPDF's sdist will already create
1297 # the required binaries, but pip will still see `requires_dist` set to
1298 # 'PyMuPDFb', so will also download and build PyMuPDFb's sdist.
1299 #
1300 log(f'Specifying dummy PyMuPDFb wheel.')
1301
1302 def get_requires_for_build_wheel(config_settings=None):
1303 return list()
1304
1305 p = pipcl.Package(
1306 'PyMuPDFb',
1307 version_b,
1308 summary = 'Dummy PyMuPDFb wheel',
1309 description = '',
1310 author = 'Artifex',
1311 author_email = 'support@artifex.com',
1312 license = 'GNU AFFERO GPL 3.0',
1313 tag_python = 'py3',
1314 )
1315
1316 else:
1317 # A normal PyMuPDF package.
1318
1319 with open( f'{g_root}/README.md', encoding='utf-8') as f:
1320 readme_p = f.read()
1321
1322 with open( f'{g_root}/READMEb.md', encoding='utf-8') as f:
1323 readme_b = f.read()
1324
1325 with open( f'{g_root}/READMEd.md', encoding='utf-8') as f:
1326 readme_d = f.read()
1327
1328 tag_python = None
1329 requires_dist = list()
1330 entry_points = None
1331
1332 if 'p' in PYMUPDF_SETUP_FLAVOUR:
1333 version = version_p
1334 name = 'PyMuPDF'
1335 readme = readme_p
1336 summary = 'A high performance Python library for data extraction, analysis, conversion & manipulation of PDF (and other) documents.'
1337 if 'b' not in PYMUPDF_SETUP_FLAVOUR:
1338 requires_dist.append(f'PyMuPDFb =={version_b}')
1339 # Create a `pymupdf` command.
1340 entry_points = textwrap.dedent('''
1341 [console_scripts]
1342 pymupdf = pymupdf.__main__:main
1343 ''')
1344 elif 'b' in PYMUPDF_SETUP_FLAVOUR:
1345 version = version_b
1346 name = 'PyMuPDFb'
1347 readme = readme_b
1348 summary = 'MuPDF shared libraries for PyMuPDF.'
1349 tag_python = 'py3'
1350 elif 'd' in PYMUPDF_SETUP_FLAVOUR:
1351 version = version_b
1352 name = 'PyMuPDFd'
1353 readme = readme_d
1354 summary = 'MuPDF build-time files for PyMuPDF.'
1355 tag_python = 'py3'
1356 else:
1357 assert 0, f'Unrecognised {PYMUPDF_SETUP_FLAVOUR=}.'
1358
1359 if os.environ.get('PYODIDE_ROOT'):
1360 # We can't pip install pytest on pyodide, so specify it here.
1361 requires_dist.append('pytest')
1362
1363 p = pipcl.Package(
1364 name,
1365 version,
1366 summary = summary,
1367 description = readme,
1368 description_content_type = 'text/markdown',
1369 classifier = classifier,
1370 author = 'Artifex',
1371 author_email = 'support@artifex.com',
1372 requires_dist = requires_dist,
1373 requires_python = '>=3.9',
1374 license = 'Dual Licensed - GNU AFFERO GPL 3.0 or Artifex Commercial License',
1375 project_url = [
1376 ('Documentation, https://pymupdf.readthedocs.io/'),
1377 ('Source, https://github.com/pymupdf/pymupdf'),
1378 ('Tracker, https://github.com/pymupdf/PyMuPDF/issues'),
1379 ('Changelog, https://pymupdf.readthedocs.io/en/latest/changes.html'),
1380 ],
1381
1382 entry_points = entry_points,
1383
1384 fn_build=build,
1385 fn_sdist=sdist,
1386
1387 tag_python=tag_python,
1388 py_limited_api=g_py_limited_api,
1389
1390 # 30MB: 9 ZIP_DEFLATED
1391 # 28MB: 9 ZIP_BZIP2
1392 # 23MB: 9 ZIP_LZMA
1393 #wheel_compression = zipfile.ZIP_DEFLATED if (darwin or pyodide) else zipfile.ZIP_LZMA,
1394 wheel_compresslevel = 9,
1395 )
1396
1397 def get_requires_for_build_wheel(config_settings=None):
1398 '''
1399 Adds to pyproject.toml:[build-system]:requires, allowing programmatic
1400 control over what packages we require.
1401 '''
1402 def platform_release_tuple():
1403 r = platform.release()
1404 r = r.split('.')
1405 r = tuple(int(i) for i in r)
1406 log(f'platform_release_tuple() returning {r=}.')
1407 return r
1408
1409 ret = list()
1410 libclang = os.environ.get('PYMUPDF_SETUP_LIBCLANG')
1411 if libclang:
1412 print(f'Overriding to use {libclang=}.')
1413 ret.append(libclang)
1414 elif openbsd:
1415 print(f'OpenBSD: libclang not available via pip; assuming `pkg_add py3-llvm`.')
1416 elif darwin and platform.machine() == 'arm64':
1417 print(f'MacOS/arm64: forcing use of libclang 16.0.6 because 18.1.1 known to fail with `clang.cindex.TranslationUnitLoadError: Error parsing translation unit.`')
1418 ret.append('libclang==16.0.6')
1419 elif darwin and platform_release_tuple() < (18,):
1420 # There are still of problems when building on old macos.
1421 ret.append('libclang==14.0.6')
1422 else:
1423 ret.append('libclang')
1424 if msys2:
1425 print(f'msys2: pip install of swig does not build; assuming `pacman -S swig`.')
1426 elif openbsd:
1427 print(f'OpenBSD: pip install of swig does not build; assuming `pkg_add swig`.')
1428 else:
1429 ret.append( 'swig')
1430 return ret
1431
1432
1433 if PYMUPDF_SETUP_URL_WHEEL:
1434 def build_wheel(
1435 wheel_directory,
1436 config_settings=None,
1437 metadata_directory=None,
1438 p=p,
1439 ):
1440 '''
1441 Instead of building wheel, we look for and copy a wheel from location
1442 specified by PYMUPDF_SETUP_URL_WHEEL.
1443 '''
1444 log(f'{PYMUPDF_SETUP_URL_WHEEL=}')
1445 log(f'{p.wheel_name()=}')
1446 url = PYMUPDF_SETUP_URL_WHEEL
1447 if url.startswith(('http://', 'https://')):
1448 leaf = p.wheel_name()
1449 out_path = f'{wheel_directory}{leaf}'
1450 out_path_temp = out_path + '-'
1451 if url.endswith('/'):
1452 url += leaf
1453 log(f'Downloading from {url=} to {out_path_temp=}.')
1454 urllib.request.urlretrieve(url, out_path_temp)
1455 elif url.startswith(f'file://'):
1456 in_path = url[len('file://'):]
1457 log(f'{in_path=}')
1458 if in_path.endswith('/'):
1459 # Look for matching wheel within this directory.
1460 wheels = glob.glob(f'{in_path}*.whl')
1461 log(f'{len(wheels)=}')
1462 for in_path in wheels:
1463 log(f'{in_path=}')
1464 leaf = os.path.basename(in_path)
1465 if p.wheel_name_match(leaf):
1466 log(f'Match: {in_path=}')
1467 break
1468 else:
1469 message = f'Cannot find matching for {p.wheel_name()=} in ({len(wheels)=}):\n'
1470 wheels_text = ''
1471 for wheel in wheels:
1472 wheels_text += f' {wheel}\n'
1473 assert 0, f'Cannot find matching for {p.wheel_name()=} in:\n{wheels_text}'
1474 else:
1475 leaf = os.path.basename(in_path)
1476 out_path = os.path.join(wheel_directory, leaf)
1477 out_path_temp = out_path + '-'
1478 log(f'Copying from {in_path=} to {out_path_temp=}.')
1479 shutil.copy2(in_path, out_path_temp)
1480 else:
1481 assert 0, f'Unrecognised prefix in {PYMUPDF_SETUP_URL_WHEEL=}.'
1482
1483 log(f'Renaming from:\n {out_path_temp}\nto:\n {out_path}.')
1484 os.rename(out_path_temp, out_path)
1485 return os.path.basename(out_path)
1486 else:
1487 build_wheel = p.build_wheel
1488
1489 build_sdist = p.build_sdist
1490
1491
1492 if __name__ == '__main__':
1493 p.handle_argv(sys.argv)