Mercurial > hgrepos > Python > libs > ConfigMix

annotate configmix/__init__.py @ 443:23941c014130

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
author Franz Glasner <fzglas.hg@dom66.de>
date Fri, 10 Dec 2021 09:27:47 +0100
parents fa08713a4992
children 7ff4ee40b36c
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
53ea2bc254e7 Begin a package to abstract some of the important configuration handling stuff.
Franz Glasner <hg@dom66.de>
parents:
diff changeset
1 # -*- coding: utf-8 -*-
82
218807d7d883 Remove header markup from the Python files and put them into the doc .rst files
Franz Glasner <hg@dom66.de>
parents: 79
diff changeset
2 """A library for helping with configuration files.
4
f76d85ccc5b9 Switch to the "New BSD License"
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 0
diff changeset
3
73
6216c561532a Put a Copyright and License notice into the package's __init__.py
Franz Glasner <hg@dom66.de>
parents: 69
diff changeset
4 :Author: Franz Glasner
290
aec97edf7945 Prepare for new dev: bump version to 0.13.dev1 and adjust copyright year for most important files
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 286
diff changeset
5 :Copyright: (c) 2015–2021, Franz Glasner.
73
6216c561532a Put a Copyright and License notice into the package's __init__.py
Franz Glasner <hg@dom66.de>
parents: 69
diff changeset
6 All rights reserved.
296
eed16a1ec8f3 Use SPDX license identifiers (either full or short) all over the package
Franz Glasner <fzglas.hg@dom66.de>
parents: 293
diff changeset
7 :License: BSD 3-Clause "New" or "Revised" License.
73
6216c561532a Put a Copyright and License notice into the package's __init__.py
Franz Glasner <hg@dom66.de>
parents: 69
diff changeset
8 See LICENSE.txt for details.
174
e2505f524ab9 Use the "@(#)" sigil in the package documentation header
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 173
diff changeset
9 :ID: @(#) $Header$
4
f76d85ccc5b9 Switch to the "New BSD License"
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 0
diff changeset
10
f76d85ccc5b9 Switch to the "New BSD License"
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 0
diff changeset
11 """
0
53ea2bc254e7 Begin a package to abstract some of the important configuration handling stuff.
Franz Glasner <hg@dom66.de>
parents:
diff changeset
12
53ea2bc254e7 Begin a package to abstract some of the important configuration handling stuff.
Franz Glasner <hg@dom66.de>
parents:
diff changeset
13 from __future__ import division, print_function, absolute_import
53ea2bc254e7 Begin a package to abstract some of the important configuration handling stuff.
Franz Glasner <hg@dom66.de>
parents:
diff changeset
14
53ea2bc254e7 Begin a package to abstract some of the important configuration handling stuff.
Franz Glasner <hg@dom66.de>
parents:
diff changeset
15
434
fa08713a4992 +++++ v0.18.1
Franz Glasner <fzglas.hg@dom66.de>
parents: 429
diff changeset
16 __version__ = "0.18.1"
0
53ea2bc254e7 Begin a package to abstract some of the important configuration handling stuff.
Franz Glasner <hg@dom66.de>
parents:
diff changeset
17
163
ff03cdf36139 The README.txt should be in the most important parts readable without keyword substitutions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
18 __revision__ = "|VCSRevision|"
ff03cdf36139 The README.txt should be in the most important parts readable without keyword substitutions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
19 __date__ = "|VCSJustDate|"
52
6c7f90dbce98 Adjusted the Copyright and change the RCS keywords in accordance with "kwarchive"
Franz Glasner <hg@dom66.de>
parents: 45
diff changeset
20
177
6dde1e344ae8 Style: put "__all__" into the meta-variables section as recommended by PEP
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 175
diff changeset
21 __all__ = ["load", "safe_load",
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
22 "set_assoc", "get_assoc", "clear_assoc",
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
23 "get_default_assoc",
180
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
24 "Configuration",
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
25 "try_determine_filemode"]
177
6dde1e344ae8 Style: put "__all__" into the meta-variables section as recommended by PEP
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 175
diff changeset
26
0
53ea2bc254e7 Begin a package to abstract some of the important configuration handling stuff.
Franz Glasner <hg@dom66.de>
parents:
diff changeset
27
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
28 import fnmatch
10
58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 9
diff changeset
29 import copy
180
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
30 import io
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
31 import os
180
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
32 import re
10
58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 9
diff changeset
33
303
2a2f5b86fe34 Move some important public constants into the .constants sub-module
Franz Glasner <fzglas.hg@dom66.de>
parents: 297
diff changeset
34 from .compat import u2fs
22
6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 13
diff changeset
35 from .config import Configuration
303
2a2f5b86fe34 Move some important public constants into the .constants sub-module
Franz Glasner <fzglas.hg@dom66.de>
parents: 297
diff changeset
36 from . import constants
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
37
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
38
172
8138d56d7cd3 ".load" and ".safe_load" get a keyword parameter "defaults" that allows the provision of a configuration dictionary with default settings
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 171
diff changeset
39 def load(*files, **kwargs):
22
6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 13
diff changeset
40 """Load the given configuration files, merge them in the given order
108
2196362c0467 Enhance documentation of "configmix.load()"
Franz Glasner <hg@dom66.de>
parents: 107
diff changeset
41 and return the resulting configuration dictionary.
2196362c0467 Enhance documentation of "configmix.load()"
Franz Glasner <hg@dom66.de>
parents: 107
diff changeset
42
269
Franz Glasner <fzglas.hg@dom66.de>
parents: 268
diff changeset
43 :param files: the filenames of the configuration files to read and merge;
274
90bbade12d8e Docu: use local lookup firstly
Franz Glasner <fzglas.hg@dom66.de>
parents: 273
diff changeset
44 if a filename starts with ``<dir>`` then the name is
269
Franz Glasner <fzglas.hg@dom66.de>
parents: 268
diff changeset
45 interpreted as directory and all files are loaded in
Franz Glasner <fzglas.hg@dom66.de>
parents: 268
diff changeset
46 sorted order (non-resursively, ignoring unknown filetypes)
173
b3ba2b0265b5 Docu: Explicitely tag "defaults" as keyword argument
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 172
diff changeset
47 :keyword defaults: optional configuration dictionary with some default
b3ba2b0265b5 Docu: Explicitely tag "defaults" as keyword argument
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 172
diff changeset
48 settings where the settings from `files` are merged
b3ba2b0265b5 Docu: Explicitely tag "defaults" as keyword argument
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 172
diff changeset
49 into
214
a35b0ca8b81f FIX: Docu: Sphinx markup
Franz Glasner <fzglas.hg@dom66.de>
parents: 211
diff changeset
50 :type defaults: dict-alike or None
221
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
51 :keyword extras: optional configuration dictionary that will applied
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
52 last
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
53
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
54 Use this for example to overwrite configuration file
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
55 settings from commandline arguments.
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
56 :type extras: dict-alike or None
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
57 :keyword strict: enable strict parsing mode for parsers that support it
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
58 (e.g. to prevent duplicate keys)
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
59 :type strict: bool
108
2196362c0467 Enhance documentation of "configmix.load()"
Franz Glasner <hg@dom66.de>
parents: 107
diff changeset
60 :returns: the configuration
2196362c0467 Enhance documentation of "configmix.load()"
Franz Glasner <hg@dom66.de>
parents: 107
diff changeset
61 :rtype: ~configmix.config.Configuration
22
6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 13
diff changeset
62
6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 13
diff changeset
63 """
172
8138d56d7cd3 ".load" and ".safe_load" get a keyword parameter "defaults" that allows the provision of a configuration dictionary with default settings
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 171
diff changeset
64 defaults = kwargs.get("defaults")
221
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
65 extras = kwargs.get("extras")
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
66 strict = kwargs.get("strict", False)
217
b869e792310e FIX: Extra merged and unsafe merges in configmix.safe_load
Franz Glasner <fzglas.hg@dom66.de>
parents: 216
diff changeset
67 if defaults is None:
220
2034da70f8fd Simplify the implementation of configmix.load() and .safe_load():
Franz Glasner <fzglas.hg@dom66.de>
parents: 217
diff changeset
68 ex = Configuration()
22
6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 13
diff changeset
69 else:
220
2034da70f8fd Simplify the implementation of configmix.load() and .safe_load():
Franz Glasner <fzglas.hg@dom66.de>
parents: 217
diff changeset
70 ex = merge(None, Configuration(defaults))
2034da70f8fd Simplify the implementation of configmix.load() and .safe_load():
Franz Glasner <fzglas.hg@dom66.de>
parents: 217
diff changeset
71 for f in files:
303
2a2f5b86fe34 Move some important public constants into the .constants sub-module
Franz Glasner <fzglas.hg@dom66.de>
parents: 297
diff changeset
72 if f.startswith(constants.DIR_PREFIX):
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
73 for f2 in _get_configuration_files_from_dir(f[5:]):
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
74 nx = _load_cfg_from_file(f2, ignore_unknown=True, strict=strict)
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
75 if nx is not None:
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
76 ex = merge(nx, ex)
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
77 else:
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
78 nx = _load_cfg_from_file(f, strict=strict)
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
79 if nx is not None:
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
80 ex = merge(nx, ex)
221
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
81 if extras:
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
82 ex = merge(Configuration(extras), ex)
217
b869e792310e FIX: Extra merged and unsafe merges in configmix.safe_load
Franz Glasner <fzglas.hg@dom66.de>
parents: 216
diff changeset
83 return Configuration(ex)
22
6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 13
diff changeset
84
6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 13
diff changeset
85
172
8138d56d7cd3 ".load" and ".safe_load" get a keyword parameter "defaults" that allows the provision of a configuration dictionary with default settings
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 171
diff changeset
86 def safe_load(*files, **kwargs):
112
c50ad93eb5dc Implemented a "safe_load()" to load with safe merging
Franz Glasner <hg@dom66.de>
parents: 111
diff changeset
87 """Analogous to :func:`load` but do merging with :func:`safe_merge`
275
e2fd8fea1a4c Docu: more local lookup
Franz Glasner <fzglas.hg@dom66.de>
parents: 274
diff changeset
88 instead of :func:`.merge`
112
c50ad93eb5dc Implemented a "safe_load()" to load with safe merging
Franz Glasner <hg@dom66.de>
parents: 111
diff changeset
89
c50ad93eb5dc Implemented a "safe_load()" to load with safe merging
Franz Glasner <hg@dom66.de>
parents: 111
diff changeset
90 """
172
8138d56d7cd3 ".load" and ".safe_load" get a keyword parameter "defaults" that allows the provision of a configuration dictionary with default settings
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 171
diff changeset
91 defaults = kwargs.get("defaults")
221
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
92 extras = kwargs.get("extras")
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
93 strict = kwargs.get("strict", False)
217
b869e792310e FIX: Extra merged and unsafe merges in configmix.safe_load
Franz Glasner <fzglas.hg@dom66.de>
parents: 216
diff changeset
94 if defaults is None:
220
2034da70f8fd Simplify the implementation of configmix.load() and .safe_load():
Franz Glasner <fzglas.hg@dom66.de>
parents: 217
diff changeset
95 ex = Configuration()
112
c50ad93eb5dc Implemented a "safe_load()" to load with safe merging
Franz Glasner <hg@dom66.de>
parents: 111
diff changeset
96 else:
220
2034da70f8fd Simplify the implementation of configmix.load() and .safe_load():
Franz Glasner <fzglas.hg@dom66.de>
parents: 217
diff changeset
97 ex = safe_merge(None, Configuration(defaults))
2034da70f8fd Simplify the implementation of configmix.load() and .safe_load():
Franz Glasner <fzglas.hg@dom66.de>
parents: 217
diff changeset
98 for f in files:
303
2a2f5b86fe34 Move some important public constants into the .constants sub-module
Franz Glasner <fzglas.hg@dom66.de>
parents: 297
diff changeset
99 if f.startswith(constants.DIR_PREFIX):
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
100 for f2 in _get_configuration_files_from_dir(f[5:]):
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
101 nx = _load_cfg_from_file(f2, ignore_unknown=True, strict=strict)
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
102 if nx is not None:
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
103 ex = safe_merge(nx, ex)
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
104 else:
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
105 nx = _load_cfg_from_file(f, strict=strict)
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
106 if nx is not None:
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
107 ex = safe_merge(nx, ex)
221
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
108 if extras:
223
90dd0d04b926 +++++ v0.7.1
Franz Glasner <fzglas.hg@dom66.de>
parents: 221
diff changeset
109 ex = safe_merge(Configuration(extras), ex)
217
b869e792310e FIX: Extra merged and unsafe merges in configmix.safe_load
Franz Glasner <fzglas.hg@dom66.de>
parents: 216
diff changeset
110 return Configuration(ex)
112
c50ad93eb5dc Implemented a "safe_load()" to load with safe merging
Franz Glasner <hg@dom66.de>
parents: 111
diff changeset
111
c50ad93eb5dc Implemented a "safe_load()" to load with safe merging
Franz Glasner <hg@dom66.de>
parents: 111
diff changeset
112
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
113 def _get_configuration_files_from_dir(root):
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
114 """Returns the sorted list of files within directory `root`
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
115
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
116 """
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
117 files = []
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
118
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
119 if not os.path.isdir(root):
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
120 return files
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
121
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
122 dirfiles = os.listdir(root)
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
123 dirfiles.sort()
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
124 for f in dirfiles:
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
125 path = os.path.join(root, f)
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
126 if os.path.isdir(path):
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
127 # XXX TBD Recurse??? depth-first???
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
128 continue
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
129 files.append(path)
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
130 return files
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
131
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
132
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
133 def _load_yaml(filename, strict=False):
138
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
134 from . import yaml
166
b5ce9a8461bf Use the filesystem encoding explicitely where appropriate.
Franz Glasner <fzglas.hg@dom66.de>
parents: 165
diff changeset
135 with open(u2fs(filename), "rb") as yf:
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
136 return yaml.safe_load(yf, strict=strict)
138
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
137
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
138
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
139 def _load_json(filename, strict=False):
138
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
140 from . import json
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
141 return json.load(filename)
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
142
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
143
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
144 def _load_py(filename, strict=False):
138
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
145 from . import py
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
146 return py.load(filename)
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
147
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
148
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
149 def _load_ini(filename, strict=False):
138
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
150 from . import ini
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
151 return ini.load(filename)
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
152
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
153
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
154 def _load_toml(filename, strict=False):
195
28e6c1413947 Added support for TOML style configuration files
Franz Glasner <fzglas.hg@dom66.de>
parents: 191
diff changeset
155 from . import toml
28e6c1413947 Added support for TOML style configuration files
Franz Glasner <fzglas.hg@dom66.de>
parents: 191
diff changeset
156 return toml.load(filename)
28e6c1413947 Added support for TOML style configuration files
Franz Glasner <fzglas.hg@dom66.de>
parents: 191
diff changeset
157
28e6c1413947 Added support for TOML style configuration files
Franz Glasner <fzglas.hg@dom66.de>
parents: 191
diff changeset
158
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
159 def _load_ignore(filename, strict=False):
227
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
160 """A loader that returns `None` just to ignore `filename`"""
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
161 return None
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
162
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
163
180
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
164 EMACS_MODELINE = re.compile(r"-\*-(.*?)-\*-")
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
165 EMACS_MODE = re.compile(r"(?:\A\s*|;\s*)mode[:=]\s*([-_.a-zA-Z0-9]+)")
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
166
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
167
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
168 def try_determine_filemode(filename):
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
169 """Try to determine an explicitely given filemode from an Emacs-compatible
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
170 mode declaration (e.g. ``mode=python``).
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
171
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
172 :param str filename:
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
173 :return: the found mode string or `None`
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
174 :rtype: str or None
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
175
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
176 Only the first two lines are searched for.
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
177
206
5064e3a2e54a Doc: "configmix.try_determine_filemode()" got additional documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 197
diff changeset
178 Conveniently to be used in calls to :func:`~.set_assoc` to determine
5064e3a2e54a Doc: "configmix.try_determine_filemode()" got additional documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 197
diff changeset
179 the file-mode by content instead of filename extension.
5064e3a2e54a Doc: "configmix.try_determine_filemode()" got additional documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 197
diff changeset
180
180
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
181 """
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
182 with io.open(filename, encoding="ascii", errors="replace") as f:
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
183 idx = 0
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
184 for l in f:
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
185 idx += 1
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
186 mo = EMACS_MODELINE.search(l)
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
187 if mo:
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
188 mo = EMACS_MODE.search(mo.group(1))
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
189 if mo:
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
190 return mo.group(1)
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
191 if idx >= 2:
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
192 break
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
193 return None
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
194
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
195
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
196 DEFAULT_MODE_LOADERS = {
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
197 "python": _load_py,
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
198 "yaml": _load_yaml,
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
199 "conf": _load_ini,
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
200 "conf-windows": _load_ini,
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
201 "ini": _load_ini,
195
28e6c1413947 Added support for TOML style configuration files
Franz Glasner <fzglas.hg@dom66.de>
parents: 191
diff changeset
202 "toml": _load_toml,
197
7865f28038a4 Add the "conf-toml" mode mapping to the TOML loader
Franz Glasner <fzglas.hg@dom66.de>
parents: 195
diff changeset
203 "conf-toml": _load_toml,
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
204 "javascript": _load_json,
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
205 "json": _load_json,
228
b2c75efad9e4 Renamed the 'ignore' loader key to '-*-ignore-*-'
Franz Glasner <fzglas.hg@dom66.de>
parents: 227
diff changeset
206 "-*-ignore-*-": _load_ignore,
268
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
207 "-*- ignore -*-": _load_ignore,
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
208 }
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
209 """Default associations between file modes and loader functions"""
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
210
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
211
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
212 DEFAULT_ASSOC = [
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
213 ("*.yml", "yaml"),
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
214 ("*.yaml", "yaml"),
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
215 ("*.json", "json"),
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
216 ("*.py", "python"),
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
217 ("*.ini", "conf"),
195
28e6c1413947 Added support for TOML style configuration files
Franz Glasner <fzglas.hg@dom66.de>
parents: 191
diff changeset
218 ("*.toml", "toml"),
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
219 ]
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
220 """The builtin default associations of filename extensions with
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
221 file modes -- in that order.
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
222
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
223 The "mode" part may be a string or a callable with a filename
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
224 parameter that returns the mode string for the file or `None` if it
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
225 can not determined.
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
226
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
227 """
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
228
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
229
188
17b938ccecb8 Rename the DEFAULT_LOADER marker to USE_DEFAULT_ASSOC
Franz Glasner <fzglas.hg@dom66.de>
parents: 187
diff changeset
230 USE_DEFAULT_ASSOC = object()
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
231 """Marker for the default association for an extension.
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
232
274
90bbade12d8e Docu: use local lookup firstly
Franz Glasner <fzglas.hg@dom66.de>
parents: 273
diff changeset
233 To be used in :func:`.set_assoc`.
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
234 """
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
235
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
236
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
237 def get_default_assoc(pattern):
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
238 """Return the default file-mode association for the :mod:`fnmatch`
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
239 pattern `pattern`.
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
240
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
241 :raises: :class:`KeyError` if the `pattern` is not found.
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
242
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
243 """
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
244 for pat, fmode in DEFAULT_ASSOC:
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
245 if pattern == pat:
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
246 return fmode
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
247 else:
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
248 raise KeyError("No loader for pattern %r" % pattern)
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
249
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
250
189
0d0980ed74cc Rename the "_mode_loaders" variable to "mode_loaders" and make is therefore a "public" item
Franz Glasner <fzglas.hg@dom66.de>
parents: 188
diff changeset
251 mode_loaders = {}
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
252 """All configured associations between file modes and loader functions.
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
253
274
90bbade12d8e Docu: use local lookup firstly
Franz Glasner <fzglas.hg@dom66.de>
parents: 273
diff changeset
254 See :data:`.DEFAULT_MODE_LOADERS`.
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
255
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
256 """
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
257
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
258 _extensions = []
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
259 """All configured assiciations of filename extensions with file modes.
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
260
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
261 See :data:`DEFAULT_ASSOC`
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
262
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
263 """
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
264
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
265
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
266 def clear_assoc():
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
267 """Remove all configured loader associations.
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
268
187
d2eb83720ad8 FIX: Docu: names of predefined constants
Franz Glasner <fzglas.hg@dom66.de>
parents: 186
diff changeset
269 The :data:`.DEFAULT_ASSOC` are **not** changed.
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
270
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
271 """
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
272 del _extensions[:]
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
273
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
274
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
275 def get_assoc(pattern):
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
276 """Return the default loader for the :mod:`fnmatch` pattern `pattern`.
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
277
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
278 :raises: :class:`KeyError` if the `pattern` is not found.
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
279
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
280 """
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
281 for pat, fmode in _extensions:
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
282 if pattern == pat:
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
283 return fmode
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
284 else:
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
285 raise KeyError("No associated file-mode for pattern %r" % pattern)
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
286
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
287
186
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
288 def set_assoc(fnpattern, mode, append=False):
179
62db35db8939 Docu: formatting
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 178
diff changeset
289 """Associate a :mod:`fnmatch` style pattern `fnpattern` with a
62db35db8939 Docu: formatting
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 178
diff changeset
290 file-mode `mode` that determines what will be called when
62db35db8939 Docu: formatting
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 178
diff changeset
291 :func:`load` encounters a file argument that matches `fnpattern`.
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
292
179
62db35db8939 Docu: formatting
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 178
diff changeset
293 :param str fnpattern: the :mod:`fnmatch` pattern to associate a loader
62db35db8939 Docu: formatting
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 178
diff changeset
294 with
186
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
295 :param mode: a mode string or a callable that accepts a `filename`
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
296 argument and returns a file-mode for the given file
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
297 (or `None`)
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
298 :type mode: str or callable
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
299
186
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
300 :keyword bool append: If `False` (which is the default) then this
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
301 function inserts the given pattern at the head position of the
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
302 currently defined associations, if `True` the pattern will be appended
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
303
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
304 The OS specific case-sensitivity behaviour of
175
327032bb0f6b Docu: wording
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 174
diff changeset
305 :func:`fnmatch.fnmatch` applies (i.e. :func:`os.path.normpath`
327032bb0f6b Docu: wording
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 174
diff changeset
306 will be called for both arguments).
141
647782859ae1 An extra hint that filename extension comparisons for loader lookup are case-insensitive
Franz Glasner <hg@dom66.de>
parents: 139
diff changeset
307
188
17b938ccecb8 Rename the DEFAULT_LOADER marker to USE_DEFAULT_ASSOC
Franz Glasner <fzglas.hg@dom66.de>
parents: 187
diff changeset
308 If `loader` is :data:`.USE_DEFAULT_ASSOC` then the default association
187
d2eb83720ad8 FIX: Docu: names of predefined constants
Franz Glasner <fzglas.hg@dom66.de>
parents: 186
diff changeset
309 from :data:`.DEFAULT_ASSOC` will be used -- if any.
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
310
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
311 """
188
17b938ccecb8 Rename the DEFAULT_LOADER marker to USE_DEFAULT_ASSOC
Franz Glasner <fzglas.hg@dom66.de>
parents: 187
diff changeset
312 if mode is USE_DEFAULT_ASSOC:
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
313 for p, m in DEFAULT_ASSOC:
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
314 if p == fnpattern:
186
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
315 if append:
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
316 _extensions.append((fnpattern, m))
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
317 else:
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
318 _extensions.insert(0, (fnpattern, m))
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
319 break
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
320 else:
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
321 raise ValueError("no DEFAULT mode for pattern: %r" % fnpattern)
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
322 else:
186
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
323 if append:
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
324 _extensions.append((fnpattern, mode))
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
325 else:
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
326 _extensions.insert(0, (fnpattern, mode))
138
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
327
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
328
268
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
329 def del_assoc(fnpattern):
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
330 """Remove all associations for `fnpattern`.
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
331
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
332 :param str fnpattern: the :mod:`fnmatch` pattern to associate a loader
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
333 with
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
334
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
335 """
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
336 while True:
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
337 for i in range(len(_extensions)):
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
338 pat, fmode = _extensions[i]
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
339 if fnpattern == pat:
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
340 del _extensions[i]
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
341 break # restart
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
342 else:
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
343 return # nothing deleted -> done
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
344
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
345
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
346 def _load_cfg_from_file(filename, ignore_unknown=False, strict=False):
227
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
347 """Determine the loader for file `filename` and return the loaded
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
348 configuration dict.
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
349
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
350 If `ignore_unknown` is `True` then unknown extensions are ignored.
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
351 Otherwise a :exc:`ValueError` exception is raised.
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
352
227
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
353 Can return `None` is the file should be ignored by the caller.
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
354
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
355 """
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
356 for p, m in _extensions:
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
357 if fnmatch.fnmatch(filename, p):
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
358 if callable(m):
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
359 m = m(filename)
265
39c379fa8c65 FIX: Check the return value when calling only when the callable has been really called.
Franz Glasner <fzglas.hg@dom66.de>
parents: 262
diff changeset
360 if m is None:
39c379fa8c65 FIX: Check the return value when calling only when the callable has been really called.
Franz Glasner <fzglas.hg@dom66.de>
parents: 262
diff changeset
361 continue
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
362 return mode_loaders[m](filename, strict=strict)
22
6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 13
diff changeset
363 else:
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
364 if ignore_unknown:
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
365 return None
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
366 else:
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
367 raise ValueError("Unknown configuration file type for filename "
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
368 "%r" % filename)
8
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
369
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
370
11
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
371 if 0:
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
372 #
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
373 # From: https://github.com/jet9/python-yconfig/blob/master/yconfig.py
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
374 # License: BSD License
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
375 #
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
376 def dict_merge(a, b):
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
377 """Recursively merges dict's. not just simple a['key'] = b['key'], if
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
378 both a and bhave a key who's value is a dict then dict_merge is called
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
379 on both values and the result stored in the returned dictionary."""
8
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
380
11
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
381 if not isinstance(b, dict):
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
382 return b
207
b3b5ed34d180 Handle most flake8 errors and warnings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
383 result = deepcopy(a) # noqa
11
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
384 for k, v in b.iteritems():
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
385 if k in result and isinstance(result[k], dict):
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
386 result[k] = dict_merge(result[k], v)
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
387 else:
207
b3b5ed34d180 Handle most flake8 errors and warnings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
388 result[k] = deepcopy(v) # noqa
11
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
389 return result
8
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
390
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
391
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
392 def merge(user, default, filter_comments=True):
110
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
393 """Logically merge the configuration in `user` into `default`.
98
d6ba53ce2091 Better documentation of the core function in "configmix"
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 96
diff changeset
394
d6ba53ce2091 Better documentation of the core function in "configmix"
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 96
diff changeset
395 :param ~configmix.config.Configuration user:
110
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
396 the new configuration that will be logically merged
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
397 into `default`
98
d6ba53ce2091 Better documentation of the core function in "configmix"
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 96
diff changeset
398 :param ~configmix.config.Configuration default:
110
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
399 the base configuration where `user` is logically merged into
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
400 :param bool filter_comments: flag whether to filter comment keys that
274
90bbade12d8e Docu: use local lookup firstly
Franz Glasner <fzglas.hg@dom66.de>
parents: 273
diff changeset
401 start with any of the items in :data:`.COMMENTS`
110
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
402 :returns: `user` with the necessary amendments from `default`.
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
403 If `user` is ``None`` then `default` is returned.
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
404
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
405 .. note:: The configuration in `user` is augmented/changed
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
406 **inplace**.
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
407
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
408 The configuration in `default` will be changed **inplace**
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
409 when filtering out comments (which is the default).
8
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
410
303
2a2f5b86fe34 Move some important public constants into the .constants sub-module
Franz Glasner <fzglas.hg@dom66.de>
parents: 297
diff changeset
411 If a value in `user` is equal to :data:`.constants.DEL_VALUE`
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
412 (``{{::DEL::}}``) the corresponding key will be deleted from the
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
413 merged output.
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
414
8
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
415 From http://stackoverflow.com/questions/823196/yaml-merge-in-python
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
416
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
417 """
110
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
418 if user is None:
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
419 if filter_comments:
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
420 _filter_comments(default)
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
421 _filter_deletions(default)
13
24ba462b9b4b Return the `default' argument when the given `user' argument is `None' and when it is the first call on merge
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 11
diff changeset
422 return default
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
423 if filter_comments:
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
424 _filter_comments(user)
8
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
425 if isinstance(user, dict) and isinstance(default, dict):
9
6835a5663008 FIX: Style
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 8
diff changeset
426 for k, v in default.items():
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
427 if filter_comments and _is_comment(k):
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
428 continue
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
429 if k in user:
303
2a2f5b86fe34 Move some important public constants into the .constants sub-module
Franz Glasner <fzglas.hg@dom66.de>
parents: 297
diff changeset
430 if user[k] == constants.DEL_VALUE:
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
431 # do not copy
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
432 del user[k]
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
433 else:
443
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
434 if isinstance(user, Configuration):
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
435 user[k] = _merge(
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
436 user.getitem_ns(k), v, filter_comments)
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
437 else:
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
438 user[k] = _merge(user[k], v, filter_comments)
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
439 else:
8
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
440 user[k] = v
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
441 _filter_deletions(user)
8
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
442 return user
10
58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 9
diff changeset
443
58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 9
diff changeset
444
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
445 def _merge(user, default, filter_comments):
275
e2fd8fea1a4c Docu: more local lookup
Franz Glasner <fzglas.hg@dom66.de>
parents: 274
diff changeset
446 """Recursion helper for :func:`.merge`
110
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
447
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
448 """
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
449 if isinstance(user, dict) and isinstance(default, dict):
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
450 for k, v in default.items():
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
451 if filter_comments and _is_comment(k):
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
452 continue
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
453 if k in user:
303
2a2f5b86fe34 Move some important public constants into the .constants sub-module
Franz Glasner <fzglas.hg@dom66.de>
parents: 297
diff changeset
454 if user[k] == constants.DEL_VALUE:
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
455 # do not copy
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
456 del user[k]
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
457 else:
443
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
458 if isinstance(user, Configuration):
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
459 user[k] = _merge(
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
460 user.getitem_ns(k), v, filter_comments)
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
461 else:
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
462 user[k] = _merge(user[k], v, filter_comments)
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
463 else:
110
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
464 user[k] = v
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
465 return user
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
466
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
467
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
468 def safe_merge(user, default, filter_comments=True):
275
e2fd8fea1a4c Docu: more local lookup
Franz Glasner <fzglas.hg@dom66.de>
parents: 274
diff changeset
469 """A more safe version of :func:`.merge` that makes deep copies of
13
24ba462b9b4b Return the `default' argument when the given `user' argument is `None' and when it is the first call on merge
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 11
diff changeset
470 the returned container objects.
24ba462b9b4b Return the `default' argument when the given `user' argument is `None' and when it is the first call on merge
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 11
diff changeset
471
275
e2fd8fea1a4c Docu: more local lookup
Franz Glasner <fzglas.hg@dom66.de>
parents: 274
diff changeset
472 Contrary to :func:`.merge` no given argument is ever changed
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
473 inplace. Every object from `default` is decoupled from the result
165
6ca90e80f4f4 FIX: Docu: wording
Franz Glasner <fzglas.hg@dom66.de>
parents: 163
diff changeset
474 -- so changing the `default` configuration later does not propagate
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
475 into a merged configuration later.
111
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
476
10
58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 9
diff changeset
477 """
110
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
478 if user is None:
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
479 if filter_comments:
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
480 _filter_comments(default)
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
481 _filter_deletions(default)
111
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
482 return copy.deepcopy(default)
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
483 user = copy.deepcopy(user)
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
484 if filter_comments:
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
485 _filter_comments(user)
10
58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 9
diff changeset
486 if isinstance(user, dict) and isinstance(default, dict):
58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 9
diff changeset
487 for k, v in default.items():
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
488 if filter_comments and _is_comment(k):
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
489 continue
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
490 if k in user:
303
2a2f5b86fe34 Move some important public constants into the .constants sub-module
Franz Glasner <fzglas.hg@dom66.de>
parents: 297
diff changeset
491 if user[k] == constants.DEL_VALUE:
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
492 # do not copy
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
493 del user[k]
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
494 else:
443
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
495 if isinstance(user, Configuration):
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
496 user[k] = _safe_merge(
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
497 user.getitem_ns(k), v, filter_comments)
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
498 else:
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
499 user[k] = _safe_merge(user[k], v, filter_comments)
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
500 else:
111
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
501 user[k] = copy.deepcopy(v)
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
502 _filter_deletions(user)
10
58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 9
diff changeset
503 return user
111
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
504
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
505
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
506 def _safe_merge(user, default, filter_comments):
274
90bbade12d8e Docu: use local lookup firstly
Franz Glasner <fzglas.hg@dom66.de>
parents: 273
diff changeset
507 """Recursion helper for :func:`safe_merge`
111
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
508
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
509 """
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
510 if isinstance(user, dict) and isinstance(default, dict):
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
511 for k, v in default.items():
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
512 if filter_comments and _is_comment(k):
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
513 continue
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
514 if k in user:
303
2a2f5b86fe34 Move some important public constants into the .constants sub-module
Franz Glasner <fzglas.hg@dom66.de>
parents: 297
diff changeset
515 if user[k] == constants.DEL_VALUE:
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
516 # do not copy
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
517 del user[k]
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
518 else:
443
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
519 if isinstance(user, Configuration):
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
520 user[k] = _safe_merge(
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
521 user.getitem_ns(k), v, filter_comments)
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
522 else:
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
523 user[k] = _safe_merge(user[k], v, filter_comments)
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
524 else:
111
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
525 user[k] = copy.deepcopy(v)
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
526 return user
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
527
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
528
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
529 def _filter_comments(d):
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
530 """Recursively filter comments keys in the dict `d`.
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
531
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
532 Comment keys are keys that start with any of the items in
303
2a2f5b86fe34 Move some important public constants into the .constants sub-module
Franz Glasner <fzglas.hg@dom66.de>
parents: 297
diff changeset
533 :data:`.constants.COMMENTS`.
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
534
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
535 """
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
536 if not isinstance(d, dict):
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
537 return
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
538 # use a copy of the keys because we change `d` while iterating
443
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
539 if isinstance(d, Configuration):
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
540 for k in list(d.keys()):
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
541 if _is_comment(k):
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
542 del d[k]
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
543 else:
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
544 dk = d.getitem_ns(k)
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
545 if isinstance(dk, dict):
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
546 _filter_comments(dk)
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
547 else:
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
548 for k in list(d.keys()):
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
549 if _is_comment(k):
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
550 del d[k]
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
551 else:
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
552 if isinstance(d[k], dict):
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
553 _filter_comments(d[k])
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
554
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
555
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
556 def _is_comment(k):
303
2a2f5b86fe34 Move some important public constants into the .constants sub-module
Franz Glasner <fzglas.hg@dom66.de>
parents: 297
diff changeset
557 for i in constants.COMMENTS:
256
7e26d31f52de FIX: Allow non-text keys when merging: handle .startswith() errors gracefully
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 253
diff changeset
558 try:
7e26d31f52de FIX: Allow non-text keys when merging: handle .startswith() errors gracefully
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 253
diff changeset
559 if k.startswith(i):
7e26d31f52de FIX: Allow non-text keys when merging: handle .startswith() errors gracefully
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 253
diff changeset
560 return True
7e26d31f52de FIX: Allow non-text keys when merging: handle .startswith() errors gracefully
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 253
diff changeset
561 except AttributeError:
7e26d31f52de FIX: Allow non-text keys when merging: handle .startswith() errors gracefully
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 253
diff changeset
562 # non-string key
7e26d31f52de FIX: Allow non-text keys when merging: handle .startswith() errors gracefully
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 253
diff changeset
563 return False
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
564 return False
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
565
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
566
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
567 def _filter_deletions(d):
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
568 """Recursively filter deletions in the dict `d`.
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
569
303
2a2f5b86fe34 Move some important public constants into the .constants sub-module
Franz Glasner <fzglas.hg@dom66.de>
parents: 297
diff changeset
570 Deletions have values that equal :data:`.constants.DEL_VALUE`.
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
571
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
572 """
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
573 if not isinstance(d, dict):
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
574 return
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
575 # use a copy of the items because we change `d` while iterating
443
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
576 if isinstance(d, Configuration):
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
577 for k, v in list(d.items()):
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
578 if v == constants.DEL_VALUE:
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
579 del d[k]
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
580 else:
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
581 dk = d.getitem_ns(k)
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
582 if isinstance(dk, dict):
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
583 _filter_deletions(dk)
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
584 else:
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
585 for k, v in list(d.items()):
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
586 if v == constants.DEL_VALUE:
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
587 del d[k]
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
588 else:
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
589 if isinstance(d[k], dict):
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
590 _filter_deletions(d[k])
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
591
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
592
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
593 #
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
594 # Init loader defaults: mode->loader and extension->mode
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
595 #
189
0d0980ed74cc Rename the "_mode_loaders" variable to "mode_loaders" and make is therefore a "public" item
Franz Glasner <fzglas.hg@dom66.de>
parents: 188
diff changeset
596 mode_loaders.update(DEFAULT_MODE_LOADERS)
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
597 for _pattern, _mode in DEFAULT_ASSOC:
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
598 set_assoc(_pattern, _mode)