Mercurial > hgrepos > Python > libs > ConfigMix

annotate configmix/__init__.py @ 741:e069797f0e36

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Implemented the new merge stragegies when merging lists: "extend" and "prepend"
author Franz Glasner <fzglas.hg@dom66.de>
date Sun, 29 Oct 2023 17:15:41 +0100
parents 324ae9a56a75
children e4fad9cdd906
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
53ea2bc254e7 Begin a package to abstract some of the important configuration handling stuff.
Franz Glasner <hg@dom66.de>
parents:
diff changeset
1 # -*- coding: utf-8 -*-
82
218807d7d883 Remove header markup from the Python files and put them into the doc .rst files
Franz Glasner <hg@dom66.de>
parents: 79
diff changeset
2 """A library for helping with configuration files.
4
f76d85ccc5b9 Switch to the "New BSD License"
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 0
diff changeset
3
73
6216c561532a Put a Copyright and License notice into the package's __init__.py
Franz Glasner <hg@dom66.de>
parents: 69
diff changeset
4 :Author: Franz Glasner
673
77d4a3e7c9e2 Extent all important copyright years to 2023
Franz Glasner <fzglas.hg@dom66.de>
parents: 669
diff changeset
5 :Copyright: (c) 2015–2023, Franz Glasner.
73
6216c561532a Put a Copyright and License notice into the package's __init__.py
Franz Glasner <hg@dom66.de>
parents: 69
diff changeset
6 All rights reserved.
296
eed16a1ec8f3 Use SPDX license identifiers (either full or short) all over the package
Franz Glasner <fzglas.hg@dom66.de>
parents: 293
diff changeset
7 :License: BSD 3-Clause "New" or "Revised" License.
73
6216c561532a Put a Copyright and License notice into the package's __init__.py
Franz Glasner <hg@dom66.de>
parents: 69
diff changeset
8 See LICENSE.txt for details.
174
e2505f524ab9 Use the "@(#)" sigil in the package documentation header
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 173
diff changeset
9 :ID: @(#) $Header$
4
f76d85ccc5b9 Switch to the "New BSD License"
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 0
diff changeset
10
f76d85ccc5b9 Switch to the "New BSD License"
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 0
diff changeset
11 """
0
53ea2bc254e7 Begin a package to abstract some of the important configuration handling stuff.
Franz Glasner <hg@dom66.de>
parents:
diff changeset
12
53ea2bc254e7 Begin a package to abstract some of the important configuration handling stuff.
Franz Glasner <hg@dom66.de>
parents:
diff changeset
13 from __future__ import division, print_function, absolute_import
53ea2bc254e7 Begin a package to abstract some of the important configuration handling stuff.
Franz Glasner <hg@dom66.de>
parents:
diff changeset
14
53ea2bc254e7 Begin a package to abstract some of the important configuration handling stuff.
Franz Glasner <hg@dom66.de>
parents:
diff changeset
15
728
450223a8cff2 +++++ v0.22
Franz Glasner <fzglas.hg@dom66.de>
parents: 699
diff changeset
16 __version__ = "0.22"
0
53ea2bc254e7 Begin a package to abstract some of the important configuration handling stuff.
Franz Glasner <hg@dom66.de>
parents:
diff changeset
17
163
ff03cdf36139 The README.txt should be in the most important parts readable without keyword substitutions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
18 __revision__ = "|VCSRevision|"
ff03cdf36139 The README.txt should be in the most important parts readable without keyword substitutions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
19 __date__ = "|VCSJustDate|"
52
6c7f90dbce98 Adjusted the Copyright and change the RCS keywords in accordance with "kwarchive"
Franz Glasner <hg@dom66.de>
parents: 45
diff changeset
20
177
6dde1e344ae8 Style: put "__all__" into the meta-variables section as recommended by PEP
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 175
diff changeset
21 __all__ = ["load", "safe_load",
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
22 "set_assoc", "get_assoc", "clear_assoc",
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
23 "get_default_assoc",
180
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
24 "Configuration",
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
25 "try_determine_filemode"]
177
6dde1e344ae8 Style: put "__all__" into the meta-variables section as recommended by PEP
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 175
diff changeset
26
0
53ea2bc254e7 Begin a package to abstract some of the important configuration handling stuff.
Franz Glasner <hg@dom66.de>
parents:
diff changeset
27
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
28 import fnmatch
10
58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 9
diff changeset
29 import copy
180
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
30 import io
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
31 import os
180
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
32 import re
10
58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 9
diff changeset
33
303
2a2f5b86fe34 Move some important public constants into the .constants sub-module
Franz Glasner <fzglas.hg@dom66.de>
parents: 297
diff changeset
34 from .compat import u2fs
539
9546d38cd3f8 Refactor: the parsing of the quoted and dot-separated path string is put into a function that handles also empty inputs properly
Franz Glasner <fzglas.hg@dom66.de>
parents: 535
diff changeset
35 from .config import Configuration, quote, unquote, pathstr2path # noqa: F401
303
2a2f5b86fe34 Move some important public constants into the .constants sub-module
Franz Glasner <fzglas.hg@dom66.de>
parents: 297
diff changeset
36 from . import constants
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
37
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
38
172
8138d56d7cd3 ".load" and ".safe_load" get a keyword parameter "defaults" that allows the provision of a configuration dictionary with default settings
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 171
diff changeset
39 def load(*files, **kwargs):
22
6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 13
diff changeset
40 """Load the given configuration files, merge them in the given order
108
2196362c0467 Enhance documentation of "configmix.load()"
Franz Glasner <hg@dom66.de>
parents: 107
diff changeset
41 and return the resulting configuration dictionary.
2196362c0467 Enhance documentation of "configmix.load()"
Franz Glasner <hg@dom66.de>
parents: 107
diff changeset
42
269
Franz Glasner <fzglas.hg@dom66.de>
parents: 268
diff changeset
43 :param files: the filenames of the configuration files to read and merge;
274
90bbade12d8e Docu: use local lookup firstly
Franz Glasner <fzglas.hg@dom66.de>
parents: 273
diff changeset
44 if a filename starts with ``<dir>`` then the name is
269
Franz Glasner <fzglas.hg@dom66.de>
parents: 268
diff changeset
45 interpreted as directory and all files are loaded in
Franz Glasner <fzglas.hg@dom66.de>
parents: 268
diff changeset
46 sorted order (non-resursively, ignoring unknown filetypes)
173
b3ba2b0265b5 Docu: Explicitely tag "defaults" as keyword argument
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 172
diff changeset
47 :keyword defaults: optional configuration dictionary with some default
b3ba2b0265b5 Docu: Explicitely tag "defaults" as keyword argument
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 172
diff changeset
48 settings where the settings from `files` are merged
b3ba2b0265b5 Docu: Explicitely tag "defaults" as keyword argument
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 172
diff changeset
49 into
214
a35b0ca8b81f FIX: Docu: Sphinx markup
Franz Glasner <fzglas.hg@dom66.de>
parents: 211
diff changeset
50 :type defaults: dict-alike or None
221
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
51 :keyword extras: optional configuration dictionary that will applied
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
52 last
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
53
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
54 Use this for example to overwrite configuration file
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
55 settings from commandline arguments.
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
56 :type extras: dict-alike or None
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
57 :keyword strict: enable strict parsing mode for parsers that support it
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
58 (e.g. to prevent duplicate keys)
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
59 :type strict: bool
741
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
60 :keyword merge_lists: When ``None`` then lists will be overwritten
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
61 by the merge process. When ``extend`` then
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
62 lists will be extended instead.
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
63 This parameter is passed to :func:`.merge`.
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
64 :type merge_lists: str or None
108
2196362c0467 Enhance documentation of "configmix.load()"
Franz Glasner <hg@dom66.de>
parents: 107
diff changeset
65 :returns: the configuration
2196362c0467 Enhance documentation of "configmix.load()"
Franz Glasner <hg@dom66.de>
parents: 107
diff changeset
66 :rtype: ~configmix.config.Configuration
22
6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 13
diff changeset
67
6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 13
diff changeset
68 """
172
8138d56d7cd3 ".load" and ".safe_load" get a keyword parameter "defaults" that allows the provision of a configuration dictionary with default settings
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 171
diff changeset
69 defaults = kwargs.get("defaults")
221
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
70 extras = kwargs.get("extras")
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
71 strict = kwargs.get("strict", False)
741
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
72 merge_lists = kwargs.get("merge_lists", None)
217
b869e792310e FIX: Extra merged and unsafe merges in configmix.safe_load
Franz Glasner <fzglas.hg@dom66.de>
parents: 216
diff changeset
73 if defaults is None:
220
2034da70f8fd Simplify the implementation of configmix.load() and .safe_load():
Franz Glasner <fzglas.hg@dom66.de>
parents: 217
diff changeset
74 ex = Configuration()
22
6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 13
diff changeset
75 else:
741
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
76 ex = merge(None, Configuration(defaults), merge_lists=merge_lists)
220
2034da70f8fd Simplify the implementation of configmix.load() and .safe_load():
Franz Glasner <fzglas.hg@dom66.de>
parents: 217
diff changeset
77 for f in files:
303
2a2f5b86fe34 Move some important public constants into the .constants sub-module
Franz Glasner <fzglas.hg@dom66.de>
parents: 297
diff changeset
78 if f.startswith(constants.DIR_PREFIX):
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
79 for f2 in _get_configuration_files_from_dir(f[5:]):
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
80 nx = _load_cfg_from_file(f2, ignore_unknown=True, strict=strict)
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
81 if nx is not None:
741
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
82 ex = merge(nx, ex, merge_lists=merge_lists)
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
83 else:
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
84 nx = _load_cfg_from_file(f, strict=strict)
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
85 if nx is not None:
741
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
86 ex = merge(nx, ex, merge_lists=merge_lists)
221
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
87 if extras:
741
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
88 ex = merge(Configuration(extras), ex, merge_lists=merge_lists)
217
b869e792310e FIX: Extra merged and unsafe merges in configmix.safe_load
Franz Glasner <fzglas.hg@dom66.de>
parents: 216
diff changeset
89 return Configuration(ex)
22
6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 13
diff changeset
90
6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 13
diff changeset
91
172
8138d56d7cd3 ".load" and ".safe_load" get a keyword parameter "defaults" that allows the provision of a configuration dictionary with default settings
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 171
diff changeset
92 def safe_load(*files, **kwargs):
112
c50ad93eb5dc Implemented a "safe_load()" to load with safe merging
Franz Glasner <hg@dom66.de>
parents: 111
diff changeset
93 """Analogous to :func:`load` but do merging with :func:`safe_merge`
275
e2fd8fea1a4c Docu: more local lookup
Franz Glasner <fzglas.hg@dom66.de>
parents: 274
diff changeset
94 instead of :func:`.merge`
112
c50ad93eb5dc Implemented a "safe_load()" to load with safe merging
Franz Glasner <hg@dom66.de>
parents: 111
diff changeset
95
c50ad93eb5dc Implemented a "safe_load()" to load with safe merging
Franz Glasner <hg@dom66.de>
parents: 111
diff changeset
96 """
172
8138d56d7cd3 ".load" and ".safe_load" get a keyword parameter "defaults" that allows the provision of a configuration dictionary with default settings
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 171
diff changeset
97 defaults = kwargs.get("defaults")
221
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
98 extras = kwargs.get("extras")
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
99 strict = kwargs.get("strict", False)
741
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
100 merge_lists = kwargs.get("merge_lists", None)
217
b869e792310e FIX: Extra merged and unsafe merges in configmix.safe_load
Franz Glasner <fzglas.hg@dom66.de>
parents: 216
diff changeset
101 if defaults is None:
220
2034da70f8fd Simplify the implementation of configmix.load() and .safe_load():
Franz Glasner <fzglas.hg@dom66.de>
parents: 217
diff changeset
102 ex = Configuration()
112
c50ad93eb5dc Implemented a "safe_load()" to load with safe merging
Franz Glasner <hg@dom66.de>
parents: 111
diff changeset
103 else:
741
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
104 ex = safe_merge(None, Configuration(defaults), merge_lists=merge_lists)
220
2034da70f8fd Simplify the implementation of configmix.load() and .safe_load():
Franz Glasner <fzglas.hg@dom66.de>
parents: 217
diff changeset
105 for f in files:
303
2a2f5b86fe34 Move some important public constants into the .constants sub-module
Franz Glasner <fzglas.hg@dom66.de>
parents: 297
diff changeset
106 if f.startswith(constants.DIR_PREFIX):
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
107 for f2 in _get_configuration_files_from_dir(f[5:]):
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
108 nx = _load_cfg_from_file(f2, ignore_unknown=True, strict=strict)
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
109 if nx is not None:
741
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
110 ex = safe_merge(nx, ex, merge_lists=merge_lists)
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
111 else:
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
112 nx = _load_cfg_from_file(f, strict=strict)
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
113 if nx is not None:
741
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
114 ex = safe_merge(nx, ex, merge_lists=merge_lists)
221
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
115 if extras:
741
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
116 ex = safe_merge(Configuration(extras), ex, merge_lists=merge_lists)
217
b869e792310e FIX: Extra merged and unsafe merges in configmix.safe_load
Franz Glasner <fzglas.hg@dom66.de>
parents: 216
diff changeset
117 return Configuration(ex)
112
c50ad93eb5dc Implemented a "safe_load()" to load with safe merging
Franz Glasner <hg@dom66.de>
parents: 111
diff changeset
118
c50ad93eb5dc Implemented a "safe_load()" to load with safe merging
Franz Glasner <hg@dom66.de>
parents: 111
diff changeset
119
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
120 def _get_configuration_files_from_dir(root):
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
121 """Returns the sorted list of files within directory `root`
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
122
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
123 """
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
124 files = []
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
125
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
126 if not os.path.isdir(root):
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
127 return files
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
128
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
129 dirfiles = os.listdir(root)
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
130 dirfiles.sort()
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
131 for f in dirfiles:
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
132 path = os.path.join(root, f)
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
133 if os.path.isdir(path):
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
134 # XXX TBD Recurse??? depth-first???
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
135 continue
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
136 files.append(path)
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
137 return files
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
138
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
139
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
140 def _load_yaml(filename, strict=False):
138
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
141 from . import yaml
166
b5ce9a8461bf Use the filesystem encoding explicitely where appropriate.
Franz Glasner <fzglas.hg@dom66.de>
parents: 165
diff changeset
142 with open(u2fs(filename), "rb") as yf:
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
143 return yaml.safe_load(yf, strict=strict)
138
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
144
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
145
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
146 def _load_json(filename, strict=False):
138
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
147 from . import json
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
148 return json.load(filename)
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
149
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
150
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
151 def _load_py(filename, strict=False):
138
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
152 from . import py
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
153 return py.load(filename)
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
154
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
155
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
156 def _load_ini(filename, strict=False):
138
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
157 from . import ini
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
158 return ini.load(filename)
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
159
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
160
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
161 def _load_toml(filename, strict=False):
195
28e6c1413947 Added support for TOML style configuration files
Franz Glasner <fzglas.hg@dom66.de>
parents: 191
diff changeset
162 from . import toml
28e6c1413947 Added support for TOML style configuration files
Franz Glasner <fzglas.hg@dom66.de>
parents: 191
diff changeset
163 return toml.load(filename)
28e6c1413947 Added support for TOML style configuration files
Franz Glasner <fzglas.hg@dom66.de>
parents: 191
diff changeset
164
28e6c1413947 Added support for TOML style configuration files
Franz Glasner <fzglas.hg@dom66.de>
parents: 191
diff changeset
165
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
166 def _load_ignore(filename, strict=False):
227
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
167 """A loader that returns `None` just to ignore `filename`"""
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
168 return None
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
169
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
170
180
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
171 EMACS_MODELINE = re.compile(r"-\*-(.*?)-\*-")
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
172 EMACS_MODE = re.compile(r"(?:\A\s*|;\s*)mode[:=]\s*([-_.a-zA-Z0-9]+)")
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
173
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
174
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
175 def try_determine_filemode(filename):
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
176 """Try to determine an explicitely given filemode from an Emacs-compatible
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
177 mode declaration (e.g. ``mode=python``).
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
178
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
179 :param str filename:
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
180 :return: the found mode string or `None`
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
181 :rtype: str or None
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
182
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
183 Only the first two lines are searched for.
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
184
206
5064e3a2e54a Doc: "configmix.try_determine_filemode()" got additional documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 197
diff changeset
185 Conveniently to be used in calls to :func:`~.set_assoc` to determine
5064e3a2e54a Doc: "configmix.try_determine_filemode()" got additional documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 197
diff changeset
186 the file-mode by content instead of filename extension.
5064e3a2e54a Doc: "configmix.try_determine_filemode()" got additional documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 197
diff changeset
187
180
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
188 """
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
189 with io.open(filename, encoding="ascii", errors="replace") as f:
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
190 idx = 0
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
191 for l in f:
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
192 idx += 1
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
193 mo = EMACS_MODELINE.search(l)
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
194 if mo:
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
195 mo = EMACS_MODE.search(mo.group(1))
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
196 if mo:
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
197 return mo.group(1)
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
198 if idx >= 2:
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
199 break
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
200 return None
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
201
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
202
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
203 DEFAULT_MODE_LOADERS = {
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
204 "python": _load_py,
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
205 "yaml": _load_yaml,
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
206 "conf": _load_ini,
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
207 "conf-windows": _load_ini,
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
208 "ini": _load_ini,
195
28e6c1413947 Added support for TOML style configuration files
Franz Glasner <fzglas.hg@dom66.de>
parents: 191
diff changeset
209 "toml": _load_toml,
197
7865f28038a4 Add the "conf-toml" mode mapping to the TOML loader
Franz Glasner <fzglas.hg@dom66.de>
parents: 195
diff changeset
210 "conf-toml": _load_toml,
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
211 "javascript": _load_json,
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
212 "json": _load_json,
228
b2c75efad9e4 Renamed the 'ignore' loader key to '-*-ignore-*-'
Franz Glasner <fzglas.hg@dom66.de>
parents: 227
diff changeset
213 "-*-ignore-*-": _load_ignore,
268
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
214 "-*- ignore -*-": _load_ignore,
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
215 }
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
216 """Default associations between file modes and loader functions"""
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
217
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
218
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
219 DEFAULT_ASSOC = [
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
220 ("*.yml", "yaml"),
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
221 ("*.yaml", "yaml"),
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
222 ("*.json", "json"),
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
223 ("*.py", "python"),
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
224 ("*.ini", "conf"),
195
28e6c1413947 Added support for TOML style configuration files
Franz Glasner <fzglas.hg@dom66.de>
parents: 191
diff changeset
225 ("*.toml", "toml"),
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
226 ]
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
227 """The builtin default associations of filename extensions with
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
228 file modes -- in that order.
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
229
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
230 The "mode" part may be a string or a callable with a filename
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
231 parameter that returns the mode string for the file or `None` if it
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
232 can not determined.
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
233
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
234 """
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
235
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
236
188
17b938ccecb8 Rename the DEFAULT_LOADER marker to USE_DEFAULT_ASSOC
Franz Glasner <fzglas.hg@dom66.de>
parents: 187
diff changeset
237 USE_DEFAULT_ASSOC = object()
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
238 """Marker for the default association for an extension.
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
239
274
90bbade12d8e Docu: use local lookup firstly
Franz Glasner <fzglas.hg@dom66.de>
parents: 273
diff changeset
240 To be used in :func:`.set_assoc`.
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
241 """
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
242
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
243
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
244 def get_default_assoc(pattern):
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
245 """Return the default file-mode association for the :mod:`fnmatch`
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
246 pattern `pattern`.
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
247
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
248 :raises: :class:`KeyError` if the `pattern` is not found.
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
249
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
250 """
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
251 for pat, fmode in DEFAULT_ASSOC:
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
252 if pattern == pat:
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
253 return fmode
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
254 else:
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
255 raise KeyError("No loader for pattern %r" % pattern)
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
256
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
257
189
0d0980ed74cc Rename the "_mode_loaders" variable to "mode_loaders" and make is therefore a "public" item
Franz Glasner <fzglas.hg@dom66.de>
parents: 188
diff changeset
258 mode_loaders = {}
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
259 """All configured associations between file modes and loader functions.
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
260
274
90bbade12d8e Docu: use local lookup firstly
Franz Glasner <fzglas.hg@dom66.de>
parents: 273
diff changeset
261 See :data:`.DEFAULT_MODE_LOADERS`.
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
262
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
263 """
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
264
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
265 _extensions = []
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
266 """All configured assiciations of filename extensions with file modes.
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
267
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
268 See :data:`DEFAULT_ASSOC`
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
269
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
270 """
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
271
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
272
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
273 def clear_assoc():
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
274 """Remove all configured loader associations.
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
275
187
d2eb83720ad8 FIX: Docu: names of predefined constants
Franz Glasner <fzglas.hg@dom66.de>
parents: 186
diff changeset
276 The :data:`.DEFAULT_ASSOC` are **not** changed.
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
277
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
278 """
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
279 del _extensions[:]
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
280
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
281
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
282 def get_assoc(pattern):
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
283 """Return the default loader for the :mod:`fnmatch` pattern `pattern`.
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
284
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
285 :raises: :class:`KeyError` if the `pattern` is not found.
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
286
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
287 """
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
288 for pat, fmode in _extensions:
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
289 if pattern == pat:
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
290 return fmode
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
291 else:
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
292 raise KeyError("No associated file-mode for pattern %r" % pattern)
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
293
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
294
186
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
295 def set_assoc(fnpattern, mode, append=False):
179
62db35db8939 Docu: formatting
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 178
diff changeset
296 """Associate a :mod:`fnmatch` style pattern `fnpattern` with a
62db35db8939 Docu: formatting
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 178
diff changeset
297 file-mode `mode` that determines what will be called when
62db35db8939 Docu: formatting
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 178
diff changeset
298 :func:`load` encounters a file argument that matches `fnpattern`.
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
299
179
62db35db8939 Docu: formatting
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 178
diff changeset
300 :param str fnpattern: the :mod:`fnmatch` pattern to associate a loader
62db35db8939 Docu: formatting
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 178
diff changeset
301 with
186
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
302 :param mode: a mode string or a callable that accepts a `filename`
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
303 argument and returns a file-mode for the given file
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
304 (or `None`)
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
305 :type mode: str or callable
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
306
186
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
307 :keyword bool append: If `False` (which is the default) then this
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
308 function inserts the given pattern at the head position of the
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
309 currently defined associations, if `True` the pattern will be appended
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
310
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
311 The OS specific case-sensitivity behaviour of
175
327032bb0f6b Docu: wording
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 174
diff changeset
312 :func:`fnmatch.fnmatch` applies (i.e. :func:`os.path.normpath`
327032bb0f6b Docu: wording
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 174
diff changeset
313 will be called for both arguments).
141
647782859ae1 An extra hint that filename extension comparisons for loader lookup are case-insensitive
Franz Glasner <hg@dom66.de>
parents: 139
diff changeset
314
188
17b938ccecb8 Rename the DEFAULT_LOADER marker to USE_DEFAULT_ASSOC
Franz Glasner <fzglas.hg@dom66.de>
parents: 187
diff changeset
315 If `loader` is :data:`.USE_DEFAULT_ASSOC` then the default association
187
d2eb83720ad8 FIX: Docu: names of predefined constants
Franz Glasner <fzglas.hg@dom66.de>
parents: 186
diff changeset
316 from :data:`.DEFAULT_ASSOC` will be used -- if any.
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
317
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
318 """
188
17b938ccecb8 Rename the DEFAULT_LOADER marker to USE_DEFAULT_ASSOC
Franz Glasner <fzglas.hg@dom66.de>
parents: 187
diff changeset
319 if mode is USE_DEFAULT_ASSOC:
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
320 for p, m in DEFAULT_ASSOC:
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
321 if p == fnpattern:
186
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
322 if append:
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
323 _extensions.append((fnpattern, m))
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
324 else:
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
325 _extensions.insert(0, (fnpattern, m))
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
326 break
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
327 else:
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
328 raise ValueError("no DEFAULT mode for pattern: %r" % fnpattern)
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
329 else:
186
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
330 if append:
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
331 _extensions.append((fnpattern, mode))
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
332 else:
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
333 _extensions.insert(0, (fnpattern, mode))
138
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
334
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
335
268
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
336 def del_assoc(fnpattern):
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
337 """Remove all associations for `fnpattern`.
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
338
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
339 :param str fnpattern: the :mod:`fnmatch` pattern to associate a loader
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
340 with
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
341
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
342 """
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
343 while True:
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
344 for i in range(len(_extensions)):
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
345 pat, fmode = _extensions[i]
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
346 if fnpattern == pat:
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
347 del _extensions[i]
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
348 break # restart
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
349 else:
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
350 return # nothing deleted -> done
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
351
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
352
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
353 def _load_cfg_from_file(filename, ignore_unknown=False, strict=False):
227
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
354 """Determine the loader for file `filename` and return the loaded
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
355 configuration dict.
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
356
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
357 If `ignore_unknown` is `True` then unknown extensions are ignored.
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
358 Otherwise a :exc:`ValueError` exception is raised.
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
359
227
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
360 Can return `None` is the file should be ignored by the caller.
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
361
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
362 """
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
363 for p, m in _extensions:
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
364 if fnmatch.fnmatch(filename, p):
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
365 if callable(m):
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
366 m = m(filename)
265
39c379fa8c65 FIX: Check the return value when calling only when the callable has been really called.
Franz Glasner <fzglas.hg@dom66.de>
parents: 262
diff changeset
367 if m is None:
39c379fa8c65 FIX: Check the return value when calling only when the callable has been really called.
Franz Glasner <fzglas.hg@dom66.de>
parents: 262
diff changeset
368 continue
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
369 return mode_loaders[m](filename, strict=strict)
22
6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 13
diff changeset
370 else:
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
371 if ignore_unknown:
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
372 return None
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
373 else:
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
374 raise ValueError("Unknown configuration file type for filename "
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
375 "%r" % filename)
8
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
376
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
377
11
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
378 if 0:
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
379 #
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
380 # From: https://github.com/jet9/python-yconfig/blob/master/yconfig.py
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
381 # License: BSD License
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
382 #
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
383 def dict_merge(a, b):
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
384 """Recursively merges dict's. not just simple a['key'] = b['key'], if
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
385 both a and bhave a key who's value is a dict then dict_merge is called
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
386 on both values and the result stored in the returned dictionary."""
8
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
387
11
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
388 if not isinstance(b, dict):
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
389 return b
207
b3b5ed34d180 Handle most flake8 errors and warnings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
390 result = deepcopy(a) # noqa
11
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
391 for k, v in b.iteritems():
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
392 if k in result and isinstance(result[k], dict):
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
393 result[k] = dict_merge(result[k], v)
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
394 else:
207
b3b5ed34d180 Handle most flake8 errors and warnings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
395 result[k] = deepcopy(v) # noqa
11
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
396 return result
8
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
397
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
398
741
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
399 def merge(user, default, filter_comments=True, merge_lists=None):
110
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
400 """Logically merge the configuration in `user` into `default`.
98
d6ba53ce2091 Better documentation of the core function in "configmix"
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 96
diff changeset
401
d6ba53ce2091 Better documentation of the core function in "configmix"
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 96
diff changeset
402 :param ~configmix.config.Configuration user:
110
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
403 the new configuration that will be logically merged
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
404 into `default`
98
d6ba53ce2091 Better documentation of the core function in "configmix"
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 96
diff changeset
405 :param ~configmix.config.Configuration default:
110
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
406 the base configuration where `user` is logically merged into
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
407 :param bool filter_comments: flag whether to filter comment keys that
274
90bbade12d8e Docu: use local lookup firstly
Franz Glasner <fzglas.hg@dom66.de>
parents: 273
diff changeset
408 start with any of the items in :data:`.COMMENTS`
741
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
409 :param merge_lists: When ``None`` then lists will be overwritten
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
410 by the merge process. When ``extend`` then
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
411 lists will be extended instead.
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
412 :type merge_lists: str or None
110
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
413 :returns: `user` with the necessary amendments from `default`.
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
414 If `user` is ``None`` then `default` is returned.
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
415
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
416 .. note:: The configuration in `user` is augmented/changed
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
417 **inplace**.
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
418
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
419 The configuration in `default` will be changed **inplace**
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
420 when filtering out comments (which is the default).
8
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
421
303
2a2f5b86fe34 Move some important public constants into the .constants sub-module
Franz Glasner <fzglas.hg@dom66.de>
parents: 297
diff changeset
422 If a value in `user` is equal to :data:`.constants.DEL_VALUE`
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
423 (``{{::DEL::}}``) the corresponding key will be deleted from the
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
424 merged output.
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
425
8
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
426 From http://stackoverflow.com/questions/823196/yaml-merge-in-python
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
427
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
428 """
110
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
429 if user is None:
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
430 if filter_comments:
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
431 _filter_comments(default)
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
432 _filter_deletions(default)
13
24ba462b9b4b Return the `default' argument when the given `user' argument is `None' and when it is the first call on merge
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 11
diff changeset
433 return default
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
434 if filter_comments:
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
435 _filter_comments(user)
8
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
436 if isinstance(user, dict) and isinstance(default, dict):
9
6835a5663008 FIX: Style
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 8
diff changeset
437 for k, v in default.items():
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
438 if filter_comments and _is_comment(k):
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
439 continue
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
440 if k in user:
638
ef485419d723 FIX: Now the merge logic does not interpolate variables in any case.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 633
diff changeset
441 if isinstance(user, Configuration):
ef485419d723 FIX: Now the merge logic does not interpolate variables in any case.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 633
diff changeset
442 ukv = user.getitem_ns(k)
ef485419d723 FIX: Now the merge logic does not interpolate variables in any case.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 633
diff changeset
443 else:
ef485419d723 FIX: Now the merge logic does not interpolate variables in any case.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 633
diff changeset
444 ukv = user[k]
ef485419d723 FIX: Now the merge logic does not interpolate variables in any case.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 633
diff changeset
445 if ukv == constants.DEL_VALUE:
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
446 # do not copy
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
447 del user[k]
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
448 else:
741
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
449 user[k] = _merge(ukv, v, filter_comments, merge_lists)
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
450 else:
8
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
451 user[k] = v
741
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
452 elif merge_lists is not None:
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
453 if merge_lists == "extend":
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
454 if isinstance(user, list) and isinstance(default, list):
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
455 for idx, value in enumerate(default):
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
456 user.insert(idx, value)
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
457 elif merge_lists == "prepend":
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
458 if isinstance(user, list) and isinstance(default, list):
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
459 user.extend(default)
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
460 else:
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
461 raise ValueError(
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
462 "unknown strategy for merge_lists: %s" % (merge_lists,))
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
463 _filter_deletions(user)
8
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
464 return user
10
58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 9
diff changeset
465
58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 9
diff changeset
466
741
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
467 def _merge(user, default, filter_comments, merge_lists):
275
e2fd8fea1a4c Docu: more local lookup
Franz Glasner <fzglas.hg@dom66.de>
parents: 274
diff changeset
468 """Recursion helper for :func:`.merge`
110
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
469
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
470 """
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
471 if isinstance(user, dict) and isinstance(default, dict):
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
472 for k, v in default.items():
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
473 if filter_comments and _is_comment(k):
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
474 continue
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
475 if k in user:
638
ef485419d723 FIX: Now the merge logic does not interpolate variables in any case.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 633
diff changeset
476 if isinstance(user, Configuration):
ef485419d723 FIX: Now the merge logic does not interpolate variables in any case.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 633
diff changeset
477 ukv = user.getitem_ns(k)
ef485419d723 FIX: Now the merge logic does not interpolate variables in any case.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 633
diff changeset
478 else:
ef485419d723 FIX: Now the merge logic does not interpolate variables in any case.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 633
diff changeset
479 ukv = user[k]
ef485419d723 FIX: Now the merge logic does not interpolate variables in any case.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 633
diff changeset
480 if ukv == constants.DEL_VALUE:
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
481 # do not copy
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
482 del user[k]
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
483 else:
741
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
484 user[k] = _merge(ukv, v, filter_comments, merge_lists)
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
485 else:
110
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
486 user[k] = v
741
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
487 elif merge_lists is not None:
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
488 if merge_lists == "extend":
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
489 if isinstance(user, list) and isinstance(default, list):
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
490 for idx, value in enumerate(default):
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
491 user.insert(idx, value)
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
492 elif merge_lists == "prepend":
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
493 if isinstance(user, list) and isinstance(default, list):
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
494 user.extend(default)
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
495 else:
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
496 raise ValueError(
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
497 "unknown strategy for merge_lists: %s" % (merge_lists,))
110
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
498 return user
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
499
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
500
741
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
501 def safe_merge(user, default, filter_comments=True, merge_lists=None):
275
e2fd8fea1a4c Docu: more local lookup
Franz Glasner <fzglas.hg@dom66.de>
parents: 274
diff changeset
502 """A more safe version of :func:`.merge` that makes deep copies of
13
24ba462b9b4b Return the `default' argument when the given `user' argument is `None' and when it is the first call on merge
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 11
diff changeset
503 the returned container objects.
24ba462b9b4b Return the `default' argument when the given `user' argument is `None' and when it is the first call on merge
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 11
diff changeset
504
275
e2fd8fea1a4c Docu: more local lookup
Franz Glasner <fzglas.hg@dom66.de>
parents: 274
diff changeset
505 Contrary to :func:`.merge` no given argument is ever changed
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
506 inplace. Every object from `default` is decoupled from the result
165
6ca90e80f4f4 FIX: Docu: wording
Franz Glasner <fzglas.hg@dom66.de>
parents: 163
diff changeset
507 -- so changing the `default` configuration later does not propagate
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
508 into a merged configuration later.
111
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
509
10
58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 9
diff changeset
510 """
110
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
511 if user is None:
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
512 if filter_comments:
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
513 _filter_comments(default)
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
514 _filter_deletions(default)
111
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
515 return copy.deepcopy(default)
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
516 user = copy.deepcopy(user)
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
517 if filter_comments:
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
518 _filter_comments(user)
10
58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 9
diff changeset
519 if isinstance(user, dict) and isinstance(default, dict):
58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 9
diff changeset
520 for k, v in default.items():
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
521 if filter_comments and _is_comment(k):
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
522 continue
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
523 if k in user:
638
ef485419d723 FIX: Now the merge logic does not interpolate variables in any case.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 633
diff changeset
524 if isinstance(user, Configuration):
ef485419d723 FIX: Now the merge logic does not interpolate variables in any case.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 633
diff changeset
525 ukv = user.getitem_ns(k)
ef485419d723 FIX: Now the merge logic does not interpolate variables in any case.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 633
diff changeset
526 else:
ef485419d723 FIX: Now the merge logic does not interpolate variables in any case.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 633
diff changeset
527 ukv = user[k]
ef485419d723 FIX: Now the merge logic does not interpolate variables in any case.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 633
diff changeset
528 if ukv == constants.DEL_VALUE:
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
529 # do not copy
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
530 del user[k]
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
531 else:
741
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
532 user[k] = _safe_merge(ukv, v, filter_comments, merge_lists)
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
533 else:
111
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
534 user[k] = copy.deepcopy(v)
741
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
535 elif merge_lists is not None:
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
536 if merge_lists == "extend":
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
537 if isinstance(user, list) and isinstance(default, list):
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
538 for idx, value in enumerate(default):
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
539 user.insert(idx, copy.deepcopy(value))
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
540 elif merge_lists == "prepend":
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
541 if isinstance(user, list) and isinstance(default, list):
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
542 user.extend(copy.deepcopy(default))
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
543 else:
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
544 raise ValueError(
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
545 "unknown strategy for merge_lists: %s" % (merge_lists,))
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
546 _filter_deletions(user)
10
58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 9
diff changeset
547 return user
111
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
548
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
549
741
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
550 def _safe_merge(user, default, filter_comments, merge_lists):
274
90bbade12d8e Docu: use local lookup firstly
Franz Glasner <fzglas.hg@dom66.de>
parents: 273
diff changeset
551 """Recursion helper for :func:`safe_merge`
111
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
552
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
553 """
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
554 if isinstance(user, dict) and isinstance(default, dict):
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
555 for k, v in default.items():
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
556 if filter_comments and _is_comment(k):
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
557 continue
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
558 if k in user:
638
ef485419d723 FIX: Now the merge logic does not interpolate variables in any case.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 633
diff changeset
559 if isinstance(user, Configuration):
ef485419d723 FIX: Now the merge logic does not interpolate variables in any case.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 633
diff changeset
560 ukv = user.getitem_ns(k)
ef485419d723 FIX: Now the merge logic does not interpolate variables in any case.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 633
diff changeset
561 else:
ef485419d723 FIX: Now the merge logic does not interpolate variables in any case.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 633
diff changeset
562 ukv = user[k]
641
226aae226b8d FIX: Remove last expanding lookup of a value when merging configurations
Franz Glasner <fzglas.hg@dom66.de>
parents: 638
diff changeset
563 if ukv == constants.DEL_VALUE:
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
564 # do not copy
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
565 del user[k]
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
566 else:
741
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
567 user[k] = _safe_merge(ukv, v, filter_comments, merge_lists)
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
568 else:
111
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
569 user[k] = copy.deepcopy(v)
741
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
570 elif merge_lists is not None:
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
571 if merge_lists == "extend":
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
572 if isinstance(user, list) and isinstance(default, list):
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
573 for idx, value in enumerate(default):
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
574 user.insert(idx, copy.deepcopy(value))
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
575 elif merge_lists == "prepend":
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
576 if isinstance(user, list) and isinstance(default, list):
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
577 user.extend(copy.deepcopy(default))
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
578 else:
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
579 raise ValueError(
e069797f0e36 Implemented the new merge stragegies when merging lists: "extend" and "prepend"
Franz Glasner <fzglas.hg@dom66.de>
parents: 736
diff changeset
580 "unknown strategy for merge_lists: %s" % (merge_lists,))
111
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
581 return user
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
582
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
583
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
584 def _filter_comments(d):
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
585 """Recursively filter comments keys in the dict `d`.
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
586
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
587 Comment keys are keys that start with any of the items in
303
2a2f5b86fe34 Move some important public constants into the .constants sub-module
Franz Glasner <fzglas.hg@dom66.de>
parents: 297
diff changeset
588 :data:`.constants.COMMENTS`.
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
589
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
590 """
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
591 if not isinstance(d, dict):
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
592 return
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
593 # use a copy of the keys because we change `d` while iterating
443
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
594 if isinstance(d, Configuration):
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
595 for k in list(d.keys()):
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
596 if _is_comment(k):
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
597 del d[k]
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
598 else:
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
599 dk = d.getitem_ns(k)
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
600 if isinstance(dk, dict):
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
601 _filter_comments(dk)
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
602 else:
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
603 for k in list(d.keys()):
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
604 if _is_comment(k):
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
605 del d[k]
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
606 else:
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
607 if isinstance(d[k], dict):
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
608 _filter_comments(d[k])
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
609
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
610
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
611 def _is_comment(k):
303
2a2f5b86fe34 Move some important public constants into the .constants sub-module
Franz Glasner <fzglas.hg@dom66.de>
parents: 297
diff changeset
612 for i in constants.COMMENTS:
256
7e26d31f52de FIX: Allow non-text keys when merging: handle .startswith() errors gracefully
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 253
diff changeset
613 try:
7e26d31f52de FIX: Allow non-text keys when merging: handle .startswith() errors gracefully
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 253
diff changeset
614 if k.startswith(i):
7e26d31f52de FIX: Allow non-text keys when merging: handle .startswith() errors gracefully
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 253
diff changeset
615 return True
7e26d31f52de FIX: Allow non-text keys when merging: handle .startswith() errors gracefully
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 253
diff changeset
616 except AttributeError:
7e26d31f52de FIX: Allow non-text keys when merging: handle .startswith() errors gracefully
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 253
diff changeset
617 # non-string key
7e26d31f52de FIX: Allow non-text keys when merging: handle .startswith() errors gracefully
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 253
diff changeset
618 return False
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
619 return False
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
620
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
621
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
622 def _filter_deletions(d):
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
623 """Recursively filter deletions in the dict `d`.
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
624
303
2a2f5b86fe34 Move some important public constants into the .constants sub-module
Franz Glasner <fzglas.hg@dom66.de>
parents: 297
diff changeset
625 Deletions have values that equal :data:`.constants.DEL_VALUE`.
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
626
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
627 """
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
628 if not isinstance(d, dict):
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
629 return
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
630 # use a copy of the items because we change `d` while iterating
443
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
631 if isinstance(d, Configuration):
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
632 for k, v in list(d.items()):
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
633 if v == constants.DEL_VALUE:
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
634 del d[k]
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
635 else:
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
636 dk = d.getitem_ns(k)
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
637 if isinstance(dk, dict):
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
638 _filter_deletions(dk)
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
639 else:
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
640 for k, v in list(d.items()):
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
641 if v == constants.DEL_VALUE:
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
642 del d[k]
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
643 else:
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
644 if isinstance(d[k], dict):
23941c014130 FIX: Merge properly when the configuration's __getitem__ do now interpolate: prohibit duplicate interpolation and interpolation while merging
Franz Glasner <fzglas.hg@dom66.de>
parents: 434
diff changeset
645 _filter_deletions(d[k])
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
646
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
647
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
648 #
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
649 # Init loader defaults: mode->loader and extension->mode
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
650 #
189
0d0980ed74cc Rename the "_mode_loaders" variable to "mode_loaders" and make is therefore a "public" item
Franz Glasner <fzglas.hg@dom66.de>
parents: 188
diff changeset
651 mode_loaders.update(DEFAULT_MODE_LOADERS)
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
652 for _pattern, _mode in DEFAULT_ASSOC:
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
653 set_assoc(_pattern, _mode)