Mercurial > hgrepos > Python > libs > ConfigMix

annotate configmix/__init__.py @ 293:6f0bf673d4ff

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it. Currently only the YAML loader understands this flag.
author Franz Glasner <f.glasner@feldmann-mg.com>
date Wed, 10 Feb 2021 15:25:55 +0100
parents aec97edf7945
children eed16a1ec8f3
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
53ea2bc254e7 Begin a package to abstract some of the important configuration handling stuff.
Franz Glasner <hg@dom66.de>
parents:
diff changeset
1 # -*- coding: utf-8 -*-
82
218807d7d883 Remove header markup from the Python files and put them into the doc .rst files
Franz Glasner <hg@dom66.de>
parents: 79
diff changeset
2 """A library for helping with configuration files.
4
f76d85ccc5b9 Switch to the "New BSD License"
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 0
diff changeset
3
73
6216c561532a Put a Copyright and License notice into the package's __init__.py
Franz Glasner <hg@dom66.de>
parents: 69
diff changeset
4 :Author: Franz Glasner
290
aec97edf7945 Prepare for new dev: bump version to 0.13.dev1 and adjust copyright year for most important files
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 286
diff changeset
5 :Copyright: (c) 2015–2021, Franz Glasner.
73
6216c561532a Put a Copyright and License notice into the package's __init__.py
Franz Glasner <hg@dom66.de>
parents: 69
diff changeset
6 All rights reserved.
78
3a8f712d27df Put copyright and license into the README.txt
Franz Glasner <hg@dom66.de>
parents: 73
diff changeset
7 :License: 3-clause BSD License.
73
6216c561532a Put a Copyright and License notice into the package's __init__.py
Franz Glasner <hg@dom66.de>
parents: 69
diff changeset
8 See LICENSE.txt for details.
174
e2505f524ab9 Use the "@(#)" sigil in the package documentation header
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 173
diff changeset
9 :ID: @(#) $Header$
4
f76d85ccc5b9 Switch to the "New BSD License"
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 0
diff changeset
10
f76d85ccc5b9 Switch to the "New BSD License"
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 0
diff changeset
11 """
0
53ea2bc254e7 Begin a package to abstract some of the important configuration handling stuff.
Franz Glasner <hg@dom66.de>
parents:
diff changeset
12
53ea2bc254e7 Begin a package to abstract some of the important configuration handling stuff.
Franz Glasner <hg@dom66.de>
parents:
diff changeset
13 from __future__ import division, print_function, absolute_import
53ea2bc254e7 Begin a package to abstract some of the important configuration handling stuff.
Franz Glasner <hg@dom66.de>
parents:
diff changeset
14
53ea2bc254e7 Begin a package to abstract some of the important configuration handling stuff.
Franz Glasner <hg@dom66.de>
parents:
diff changeset
15
290
aec97edf7945 Prepare for new dev: bump version to 0.13.dev1 and adjust copyright year for most important files
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 286
diff changeset
16 __version__ = "0.13.dev1"
0
53ea2bc254e7 Begin a package to abstract some of the important configuration handling stuff.
Franz Glasner <hg@dom66.de>
parents:
diff changeset
17
163
ff03cdf36139 The README.txt should be in the most important parts readable without keyword substitutions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
18 __revision__ = "|VCSRevision|"
ff03cdf36139 The README.txt should be in the most important parts readable without keyword substitutions.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
19 __date__ = "|VCSJustDate|"
52
6c7f90dbce98 Adjusted the Copyright and change the RCS keywords in accordance with "kwarchive"
Franz Glasner <hg@dom66.de>
parents: 45
diff changeset
20
177
6dde1e344ae8 Style: put "__all__" into the meta-variables section as recommended by PEP
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 175
diff changeset
21 __all__ = ["load", "safe_load",
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
22 "set_assoc", "get_assoc", "clear_assoc",
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
23 "get_default_assoc",
180
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
24 "Configuration",
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
25 "try_determine_filemode"]
177
6dde1e344ae8 Style: put "__all__" into the meta-variables section as recommended by PEP
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 175
diff changeset
26
0
53ea2bc254e7 Begin a package to abstract some of the important configuration handling stuff.
Franz Glasner <hg@dom66.de>
parents:
diff changeset
27
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
28 import fnmatch
10
58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 9
diff changeset
29 import copy
180
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
30 import io
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
31 import os
180
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
32 import re
10
58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 9
diff changeset
33
166
b5ce9a8461bf Use the filesystem encoding explicitely where appropriate.
Franz Glasner <fzglas.hg@dom66.de>
parents: 165
diff changeset
34 from .compat import u, u2fs
22
6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 13
diff changeset
35 from .config import Configuration
10
58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 9
diff changeset
36
22
6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 13
diff changeset
37
207
b3b5ed34d180 Handle most flake8 errors and warnings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
38 COMMENTS = [
b3b5ed34d180 Handle most flake8 errors and warnings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
39 u("__comment"),
b3b5ed34d180 Handle most flake8 errors and warnings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
40 u("__doc"),
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
41 ]
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
42 """Prefixes for comment configuration keys that are to be handled as
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
43 comments
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
44
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
45 """
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
46
273
9733aaa261ac FIX: for PY2: test with the Unicode string variant when checking for "<dir>" prefixes
Franz Glasner <fzglas.hg@dom66.de>
parents: 270
diff changeset
47 DIR_PREFIX = u("<dir>")
9733aaa261ac FIX: for PY2: test with the Unicode string variant when checking for "<dir>" prefixes
Franz Glasner <fzglas.hg@dom66.de>
parents: 270
diff changeset
48 """Prefix for configuration values to read other configuration files from
9733aaa261ac FIX: for PY2: test with the Unicode string variant when checking for "<dir>" prefixes
Franz Glasner <fzglas.hg@dom66.de>
parents: 270
diff changeset
49 given directory
9733aaa261ac FIX: for PY2: test with the Unicode string variant when checking for "<dir>" prefixes
Franz Glasner <fzglas.hg@dom66.de>
parents: 270
diff changeset
50
9733aaa261ac FIX: for PY2: test with the Unicode string variant when checking for "<dir>" prefixes
Franz Glasner <fzglas.hg@dom66.de>
parents: 270
diff changeset
51 """
9733aaa261ac FIX: for PY2: test with the Unicode string variant when checking for "<dir>" prefixes
Franz Glasner <fzglas.hg@dom66.de>
parents: 270
diff changeset
52
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
53 DEL_VALUE = u("{{::DEL::}}")
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
54 """Value for configuration items to signal that the corresponding
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
55 key-value is to be deleted when configurations are merged
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
56
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
57 """
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
58
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
59
172
8138d56d7cd3 ".load" and ".safe_load" get a keyword parameter "defaults" that allows the provision of a configuration dictionary with default settings
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 171
diff changeset
60 def load(*files, **kwargs):
22
6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 13
diff changeset
61 """Load the given configuration files, merge them in the given order
108
2196362c0467 Enhance documentation of "configmix.load()"
Franz Glasner <hg@dom66.de>
parents: 107
diff changeset
62 and return the resulting configuration dictionary.
2196362c0467 Enhance documentation of "configmix.load()"
Franz Glasner <hg@dom66.de>
parents: 107
diff changeset
63
269
Franz Glasner <fzglas.hg@dom66.de>
parents: 268
diff changeset
64 :param files: the filenames of the configuration files to read and merge;
274
90bbade12d8e Docu: use local lookup firstly
Franz Glasner <fzglas.hg@dom66.de>
parents: 273
diff changeset
65 if a filename starts with ``<dir>`` then the name is
269
Franz Glasner <fzglas.hg@dom66.de>
parents: 268
diff changeset
66 interpreted as directory and all files are loaded in
Franz Glasner <fzglas.hg@dom66.de>
parents: 268
diff changeset
67 sorted order (non-resursively, ignoring unknown filetypes)
173
b3ba2b0265b5 Docu: Explicitely tag "defaults" as keyword argument
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 172
diff changeset
68 :keyword defaults: optional configuration dictionary with some default
b3ba2b0265b5 Docu: Explicitely tag "defaults" as keyword argument
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 172
diff changeset
69 settings where the settings from `files` are merged
b3ba2b0265b5 Docu: Explicitely tag "defaults" as keyword argument
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 172
diff changeset
70 into
214
a35b0ca8b81f FIX: Docu: Sphinx markup
Franz Glasner <fzglas.hg@dom66.de>
parents: 211
diff changeset
71 :type defaults: dict-alike or None
221
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
72 :keyword extras: optional configuration dictionary that will applied
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
73 last
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
74
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
75 Use this for example to overwrite configuration file
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
76 settings from commandline arguments.
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
77 :type extras: dict-alike or None
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
78 :keyword strict: enable strict parsing mode for parsers that support it
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
79 (e.g. to prevent duplicate keys)
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
80 :type strict: bool
108
2196362c0467 Enhance documentation of "configmix.load()"
Franz Glasner <hg@dom66.de>
parents: 107
diff changeset
81 :returns: the configuration
2196362c0467 Enhance documentation of "configmix.load()"
Franz Glasner <hg@dom66.de>
parents: 107
diff changeset
82 :rtype: ~configmix.config.Configuration
22
6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 13
diff changeset
83
6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 13
diff changeset
84 """
172
8138d56d7cd3 ".load" and ".safe_load" get a keyword parameter "defaults" that allows the provision of a configuration dictionary with default settings
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 171
diff changeset
85 defaults = kwargs.get("defaults")
221
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
86 extras = kwargs.get("extras")
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
87 strict = kwargs.get("strict", False)
217
b869e792310e FIX: Extra merged and unsafe merges in configmix.safe_load
Franz Glasner <fzglas.hg@dom66.de>
parents: 216
diff changeset
88 if defaults is None:
220
2034da70f8fd Simplify the implementation of configmix.load() and .safe_load():
Franz Glasner <fzglas.hg@dom66.de>
parents: 217
diff changeset
89 ex = Configuration()
22
6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 13
diff changeset
90 else:
220
2034da70f8fd Simplify the implementation of configmix.load() and .safe_load():
Franz Glasner <fzglas.hg@dom66.de>
parents: 217
diff changeset
91 ex = merge(None, Configuration(defaults))
2034da70f8fd Simplify the implementation of configmix.load() and .safe_load():
Franz Glasner <fzglas.hg@dom66.de>
parents: 217
diff changeset
92 for f in files:
273
9733aaa261ac FIX: for PY2: test with the Unicode string variant when checking for "<dir>" prefixes
Franz Glasner <fzglas.hg@dom66.de>
parents: 270
diff changeset
93 if f.startswith(DIR_PREFIX):
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
94 for f2 in _get_configuration_files_from_dir(f[5:]):
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
95 nx = _load_cfg_from_file(f2, ignore_unknown=True, strict=strict)
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
96 if nx is not None:
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
97 ex = merge(nx, ex)
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
98 else:
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
99 nx = _load_cfg_from_file(f, strict=strict)
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
100 if nx is not None:
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
101 ex = merge(nx, ex)
221
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
102 if extras:
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
103 ex = merge(Configuration(extras), ex)
217
b869e792310e FIX: Extra merged and unsafe merges in configmix.safe_load
Franz Glasner <fzglas.hg@dom66.de>
parents: 216
diff changeset
104 return Configuration(ex)
22
6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 13
diff changeset
105
6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 13
diff changeset
106
172
8138d56d7cd3 ".load" and ".safe_load" get a keyword parameter "defaults" that allows the provision of a configuration dictionary with default settings
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 171
diff changeset
107 def safe_load(*files, **kwargs):
112
c50ad93eb5dc Implemented a "safe_load()" to load with safe merging
Franz Glasner <hg@dom66.de>
parents: 111
diff changeset
108 """Analogous to :func:`load` but do merging with :func:`safe_merge`
275
e2fd8fea1a4c Docu: more local lookup
Franz Glasner <fzglas.hg@dom66.de>
parents: 274
diff changeset
109 instead of :func:`.merge`
112
c50ad93eb5dc Implemented a "safe_load()" to load with safe merging
Franz Glasner <hg@dom66.de>
parents: 111
diff changeset
110
c50ad93eb5dc Implemented a "safe_load()" to load with safe merging
Franz Glasner <hg@dom66.de>
parents: 111
diff changeset
111 """
172
8138d56d7cd3 ".load" and ".safe_load" get a keyword parameter "defaults" that allows the provision of a configuration dictionary with default settings
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 171
diff changeset
112 defaults = kwargs.get("defaults")
221
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
113 extras = kwargs.get("extras")
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
114 strict = kwargs.get("strict", False)
217
b869e792310e FIX: Extra merged and unsafe merges in configmix.safe_load
Franz Glasner <fzglas.hg@dom66.de>
parents: 216
diff changeset
115 if defaults is None:
220
2034da70f8fd Simplify the implementation of configmix.load() and .safe_load():
Franz Glasner <fzglas.hg@dom66.de>
parents: 217
diff changeset
116 ex = Configuration()
112
c50ad93eb5dc Implemented a "safe_load()" to load with safe merging
Franz Glasner <hg@dom66.de>
parents: 111
diff changeset
117 else:
220
2034da70f8fd Simplify the implementation of configmix.load() and .safe_load():
Franz Glasner <fzglas.hg@dom66.de>
parents: 217
diff changeset
118 ex = safe_merge(None, Configuration(defaults))
2034da70f8fd Simplify the implementation of configmix.load() and .safe_load():
Franz Glasner <fzglas.hg@dom66.de>
parents: 217
diff changeset
119 for f in files:
273
9733aaa261ac FIX: for PY2: test with the Unicode string variant when checking for "<dir>" prefixes
Franz Glasner <fzglas.hg@dom66.de>
parents: 270
diff changeset
120 if f.startswith(DIR_PREFIX):
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
121 for f2 in _get_configuration_files_from_dir(f[5:]):
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
122 nx = _load_cfg_from_file(f2, ignore_unknown=True, strict=strict)
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
123 if nx is not None:
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
124 ex = safe_merge(nx, ex)
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
125 else:
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
126 nx = _load_cfg_from_file(f, strict=strict)
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
127 if nx is not None:
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
128 ex = safe_merge(nx, ex)
221
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary
Franz Glasner <fzglas.hg@dom66.de>
parents: 220
diff changeset
129 if extras:
223
90dd0d04b926 +++++ v0.7.1
Franz Glasner <fzglas.hg@dom66.de>
parents: 221
diff changeset
130 ex = safe_merge(Configuration(extras), ex)
217
b869e792310e FIX: Extra merged and unsafe merges in configmix.safe_load
Franz Glasner <fzglas.hg@dom66.de>
parents: 216
diff changeset
131 return Configuration(ex)
112
c50ad93eb5dc Implemented a "safe_load()" to load with safe merging
Franz Glasner <hg@dom66.de>
parents: 111
diff changeset
132
c50ad93eb5dc Implemented a "safe_load()" to load with safe merging
Franz Glasner <hg@dom66.de>
parents: 111
diff changeset
133
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
134 def _get_configuration_files_from_dir(root):
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
135 """Returns the sorted list of files within directory `root`
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
136
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
137 """
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
138 files = []
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
139
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
140 if not os.path.isdir(root):
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
141 return files
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
142
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
143 dirfiles = os.listdir(root)
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
144 dirfiles.sort()
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
145 for f in dirfiles:
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
146 path = os.path.join(root, f)
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
147 if os.path.isdir(path):
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
148 # XXX TBD Recurse??? depth-first???
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
149 continue
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
150 files.append(path)
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
151 return files
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
152
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
153
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
154 def _load_yaml(filename, strict=False):
138
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
155 from . import yaml
166
b5ce9a8461bf Use the filesystem encoding explicitely where appropriate.
Franz Glasner <fzglas.hg@dom66.de>
parents: 165
diff changeset
156 with open(u2fs(filename), "rb") as yf:
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
157 return yaml.safe_load(yf, strict=strict)
138
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
158
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
159
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
160 def _load_json(filename, strict=False):
138
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
161 from . import json
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
162 return json.load(filename)
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
163
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
164
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
165 def _load_py(filename, strict=False):
138
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
166 from . import py
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
167 return py.load(filename)
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
168
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
169
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
170 def _load_ini(filename, strict=False):
138
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
171 from . import ini
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
172 return ini.load(filename)
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
173
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
174
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
175 def _load_toml(filename, strict=False):
195
28e6c1413947 Added support for TOML style configuration files
Franz Glasner <fzglas.hg@dom66.de>
parents: 191
diff changeset
176 from . import toml
28e6c1413947 Added support for TOML style configuration files
Franz Glasner <fzglas.hg@dom66.de>
parents: 191
diff changeset
177 return toml.load(filename)
28e6c1413947 Added support for TOML style configuration files
Franz Glasner <fzglas.hg@dom66.de>
parents: 191
diff changeset
178
28e6c1413947 Added support for TOML style configuration files
Franz Glasner <fzglas.hg@dom66.de>
parents: 191
diff changeset
179
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
180 def _load_ignore(filename, strict=False):
227
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
181 """A loader that returns `None` just to ignore `filename`"""
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
182 return None
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
183
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
184
180
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
185 EMACS_MODELINE = re.compile(r"-\*-(.*?)-\*-")
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
186 EMACS_MODE = re.compile(r"(?:\A\s*|;\s*)mode[:=]\s*([-_.a-zA-Z0-9]+)")
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
187
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
188
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
189 def try_determine_filemode(filename):
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
190 """Try to determine an explicitely given filemode from an Emacs-compatible
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
191 mode declaration (e.g. ``mode=python``).
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
192
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
193 :param str filename:
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
194 :return: the found mode string or `None`
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
195 :rtype: str or None
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
196
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
197 Only the first two lines are searched for.
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
198
206
5064e3a2e54a Doc: "configmix.try_determine_filemode()" got additional documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 197
diff changeset
199 Conveniently to be used in calls to :func:`~.set_assoc` to determine
5064e3a2e54a Doc: "configmix.try_determine_filemode()" got additional documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 197
diff changeset
200 the file-mode by content instead of filename extension.
5064e3a2e54a Doc: "configmix.try_determine_filemode()" got additional documentation
Franz Glasner <fzglas.hg@dom66.de>
parents: 197
diff changeset
201
180
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
202 """
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
203 with io.open(filename, encoding="ascii", errors="replace") as f:
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
204 idx = 0
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
205 for l in f:
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
206 idx += 1
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
207 mo = EMACS_MODELINE.search(l)
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
208 if mo:
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
209 mo = EMACS_MODE.search(mo.group(1))
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
210 if mo:
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
211 return mo.group(1)
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
212 if idx >= 2:
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
213 break
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
214 return None
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
215
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration
Franz Glasner <fzglas.hg@dom66.de>
parents: 179
diff changeset
216
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
217 DEFAULT_MODE_LOADERS = {
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
218 "python": _load_py,
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
219 "yaml": _load_yaml,
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
220 "conf": _load_ini,
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
221 "conf-windows": _load_ini,
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
222 "ini": _load_ini,
195
28e6c1413947 Added support for TOML style configuration files
Franz Glasner <fzglas.hg@dom66.de>
parents: 191
diff changeset
223 "toml": _load_toml,
197
7865f28038a4 Add the "conf-toml" mode mapping to the TOML loader
Franz Glasner <fzglas.hg@dom66.de>
parents: 195
diff changeset
224 "conf-toml": _load_toml,
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
225 "javascript": _load_json,
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
226 "json": _load_json,
228
b2c75efad9e4 Renamed the 'ignore' loader key to '-*-ignore-*-'
Franz Glasner <fzglas.hg@dom66.de>
parents: 227
diff changeset
227 "-*-ignore-*-": _load_ignore,
268
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
228 "-*- ignore -*-": _load_ignore,
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
229 }
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
230 """Default associations between file modes and loader functions"""
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
231
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
232
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
233 DEFAULT_ASSOC = [
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
234 ("*.yml", "yaml"),
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
235 ("*.yaml", "yaml"),
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
236 ("*.json", "json"),
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
237 ("*.py", "python"),
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
238 ("*.ini", "conf"),
195
28e6c1413947 Added support for TOML style configuration files
Franz Glasner <fzglas.hg@dom66.de>
parents: 191
diff changeset
239 ("*.toml", "toml"),
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
240 ]
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
241 """The builtin default associations of filename extensions with
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
242 file modes -- in that order.
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
243
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
244 The "mode" part may be a string or a callable with a filename
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
245 parameter that returns the mode string for the file or `None` if it
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
246 can not determined.
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
247
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
248 """
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
249
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
250
188
17b938ccecb8 Rename the DEFAULT_LOADER marker to USE_DEFAULT_ASSOC
Franz Glasner <fzglas.hg@dom66.de>
parents: 187
diff changeset
251 USE_DEFAULT_ASSOC = object()
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
252 """Marker for the default association for an extension.
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
253
274
90bbade12d8e Docu: use local lookup firstly
Franz Glasner <fzglas.hg@dom66.de>
parents: 273
diff changeset
254 To be used in :func:`.set_assoc`.
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
255 """
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
256
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
257
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
258 def get_default_assoc(pattern):
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
259 """Return the default file-mode association for the :mod:`fnmatch`
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
260 pattern `pattern`.
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
261
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
262 :raises: :class:`KeyError` if the `pattern` is not found.
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
263
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
264 """
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
265 for pat, fmode in DEFAULT_ASSOC:
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
266 if pattern == pat:
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
267 return fmode
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
268 else:
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
269 raise KeyError("No loader for pattern %r" % pattern)
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
270
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
271
189
0d0980ed74cc Rename the "_mode_loaders" variable to "mode_loaders" and make is therefore a "public" item
Franz Glasner <fzglas.hg@dom66.de>
parents: 188
diff changeset
272 mode_loaders = {}
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
273 """All configured associations between file modes and loader functions.
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
274
274
90bbade12d8e Docu: use local lookup firstly
Franz Glasner <fzglas.hg@dom66.de>
parents: 273
diff changeset
275 See :data:`.DEFAULT_MODE_LOADERS`.
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
276
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
277 """
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
278
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
279 _extensions = []
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
280 """All configured assiciations of filename extensions with file modes.
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
281
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
282 See :data:`DEFAULT_ASSOC`
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
283
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
284 """
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
285
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
286
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
287 def clear_assoc():
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
288 """Remove all configured loader associations.
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
289
187
d2eb83720ad8 FIX: Docu: names of predefined constants
Franz Glasner <fzglas.hg@dom66.de>
parents: 186
diff changeset
290 The :data:`.DEFAULT_ASSOC` are **not** changed.
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
291
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
292 """
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
293 del _extensions[:]
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
294
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
295
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
296 def get_assoc(pattern):
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
297 """Return the default loader for the :mod:`fnmatch` pattern `pattern`.
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
298
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
299 :raises: :class:`KeyError` if the `pattern` is not found.
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
300
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
301 """
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
302 for pat, fmode in _extensions:
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
303 if pattern == pat:
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
304 return fmode
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
305 else:
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
306 raise KeyError("No associated file-mode for pattern %r" % pattern)
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
307
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
308
186
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
309 def set_assoc(fnpattern, mode, append=False):
179
62db35db8939 Docu: formatting
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 178
diff changeset
310 """Associate a :mod:`fnmatch` style pattern `fnpattern` with a
62db35db8939 Docu: formatting
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 178
diff changeset
311 file-mode `mode` that determines what will be called when
62db35db8939 Docu: formatting
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 178
diff changeset
312 :func:`load` encounters a file argument that matches `fnpattern`.
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
313
179
62db35db8939 Docu: formatting
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 178
diff changeset
314 :param str fnpattern: the :mod:`fnmatch` pattern to associate a loader
62db35db8939 Docu: formatting
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 178
diff changeset
315 with
186
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
316 :param mode: a mode string or a callable that accepts a `filename`
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
317 argument and returns a file-mode for the given file
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
318 (or `None`)
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
319 :type mode: str or callable
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
320
186
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
321 :keyword bool append: If `False` (which is the default) then this
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
322 function inserts the given pattern at the head position of the
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
323 currently defined associations, if `True` the pattern will be appended
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
324
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
325 The OS specific case-sensitivity behaviour of
175
327032bb0f6b Docu: wording
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 174
diff changeset
326 :func:`fnmatch.fnmatch` applies (i.e. :func:`os.path.normpath`
327032bb0f6b Docu: wording
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 174
diff changeset
327 will be called for both arguments).
141
647782859ae1 An extra hint that filename extension comparisons for loader lookup are case-insensitive
Franz Glasner <hg@dom66.de>
parents: 139
diff changeset
328
188
17b938ccecb8 Rename the DEFAULT_LOADER marker to USE_DEFAULT_ASSOC
Franz Glasner <fzglas.hg@dom66.de>
parents: 187
diff changeset
329 If `loader` is :data:`.USE_DEFAULT_ASSOC` then the default association
187
d2eb83720ad8 FIX: Docu: names of predefined constants
Franz Glasner <fzglas.hg@dom66.de>
parents: 186
diff changeset
330 from :data:`.DEFAULT_ASSOC` will be used -- if any.
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
331
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
332 """
188
17b938ccecb8 Rename the DEFAULT_LOADER marker to USE_DEFAULT_ASSOC
Franz Glasner <fzglas.hg@dom66.de>
parents: 187
diff changeset
333 if mode is USE_DEFAULT_ASSOC:
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
334 for p, m in DEFAULT_ASSOC:
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
335 if p == fnpattern:
186
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
336 if append:
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
337 _extensions.append((fnpattern, m))
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
338 else:
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
339 _extensions.insert(0, (fnpattern, m))
171
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
340 break
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 170
diff changeset
341 else:
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
342 raise ValueError("no DEFAULT mode for pattern: %r" % fnpattern)
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
343 else:
186
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
344 if append:
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
345 _extensions.append((fnpattern, mode))
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
346 else:
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 181
diff changeset
347 _extensions.insert(0, (fnpattern, mode))
138
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
348
b883f4ef1967 Indirectly map extensions to configuration file styles
Franz Glasner <hg@dom66.de>
parents: 122
diff changeset
349
268
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
350 def del_assoc(fnpattern):
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
351 """Remove all associations for `fnpattern`.
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
352
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
353 :param str fnpattern: the :mod:`fnmatch` pattern to associate a loader
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
354 with
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
355
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
356 """
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
357 while True:
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
358 for i in range(len(_extensions)):
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
359 pat, fmode = _extensions[i]
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
360 if fnpattern == pat:
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
361 del _extensions[i]
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
362 break # restart
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
363 else:
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
364 return # nothing deleted -> done
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
365
1484f6c0223a Implemented "del_assoc()"
Franz Glasner <fzglas.hg@dom66.de>
parents: 267
diff changeset
366
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
367 def _load_cfg_from_file(filename, ignore_unknown=False, strict=False):
227
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
368 """Determine the loader for file `filename` and return the loaded
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
369 configuration dict.
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
370
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
371 If `ignore_unknown` is `True` then unknown extensions are ignored.
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
372 Otherwise a :exc:`ValueError` exception is raised.
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
373
227
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
374 Can return `None` is the file should be ignored by the caller.
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
375
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file
Franz Glasner <fzglas.hg@dom66.de>
parents: 226
diff changeset
376 """
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
377 for p, m in _extensions:
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
378 if fnmatch.fnmatch(filename, p):
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
379 if callable(m):
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
380 m = m(filename)
265
39c379fa8c65 FIX: Check the return value when calling only when the callable has been really called.
Franz Glasner <fzglas.hg@dom66.de>
parents: 262
diff changeset
381 if m is None:
39c379fa8c65 FIX: Check the return value when calling only when the callable has been really called.
Franz Glasner <fzglas.hg@dom66.de>
parents: 262
diff changeset
382 continue
293
6f0bf673d4ff Provide an optional "strict" flag to the top-level loader to pass it to low-level loaders that understand it.
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 290
diff changeset
383 return mode_loaders[m](filename, strict=strict)
22
6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 13
diff changeset
384 else:
266
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
385 if ignore_unknown:
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
386 return None
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
387 else:
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
388 raise ValueError("Unknown configuration file type for filename "
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames.
Franz Glasner <fzglas.hg@dom66.de>
parents: 265
diff changeset
389 "%r" % filename)
8
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
390
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
391
11
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
392 if 0:
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
393 #
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
394 # From: https://github.com/jet9/python-yconfig/blob/master/yconfig.py
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
395 # License: BSD License
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
396 #
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
397 def dict_merge(a, b):
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
398 """Recursively merges dict's. not just simple a['key'] = b['key'], if
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
399 both a and bhave a key who's value is a dict then dict_merge is called
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
400 on both values and the result stored in the returned dictionary."""
8
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
401
11
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
402 if not isinstance(b, dict):
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
403 return b
207
b3b5ed34d180 Handle most flake8 errors and warnings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
404 result = deepcopy(a) # noqa
11
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
405 for k, v in b.iteritems():
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
406 if k in result and isinstance(result[k], dict):
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
407 result[k] = dict_merge(result[k], v)
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
408 else:
207
b3b5ed34d180 Handle most flake8 errors and warnings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 206
diff changeset
409 result[k] = deepcopy(v) # noqa
11
aecb36d4025f Deactivate the "dict_merge()" function from yconfig
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 10
diff changeset
410 return result
8
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
411
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
412
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
413 def merge(user, default, filter_comments=True):
110
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
414 """Logically merge the configuration in `user` into `default`.
98
d6ba53ce2091 Better documentation of the core function in "configmix"
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 96
diff changeset
415
d6ba53ce2091 Better documentation of the core function in "configmix"
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 96
diff changeset
416 :param ~configmix.config.Configuration user:
110
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
417 the new configuration that will be logically merged
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
418 into `default`
98
d6ba53ce2091 Better documentation of the core function in "configmix"
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 96
diff changeset
419 :param ~configmix.config.Configuration default:
110
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
420 the base configuration where `user` is logically merged into
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
421 :param bool filter_comments: flag whether to filter comment keys that
274
90bbade12d8e Docu: use local lookup firstly
Franz Glasner <fzglas.hg@dom66.de>
parents: 273
diff changeset
422 start with any of the items in :data:`.COMMENTS`
110
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
423 :returns: `user` with the necessary amendments from `default`.
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
424 If `user` is ``None`` then `default` is returned.
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
425
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
426 .. note:: The configuration in `user` is augmented/changed
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
427 **inplace**.
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
428
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
429 The configuration in `default` will be changed **inplace**
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
430 when filtering out comments (which is the default).
8
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
431
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
432 If a value in `user` is equal to :data:`.DEL_VALUE`
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
433 (``{{::DEL::}}``) the corresponding key will be deleted from the
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
434 merged output.
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
435
8
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
436 From http://stackoverflow.com/questions/823196/yaml-merge-in-python
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
437
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
438 """
110
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
439 if user is None:
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
440 if filter_comments:
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
441 _filter_comments(default)
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
442 _filter_deletions(default)
13
24ba462b9b4b Return the `default' argument when the given `user' argument is `None' and when it is the first call on merge
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 11
diff changeset
443 return default
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
444 if filter_comments:
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
445 _filter_comments(user)
8
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
446 if isinstance(user, dict) and isinstance(default, dict):
9
6835a5663008 FIX: Style
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 8
diff changeset
447 for k, v in default.items():
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
448 if filter_comments and _is_comment(k):
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
449 continue
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
450 if k in user:
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
451 if user[k] == DEL_VALUE:
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
452 # do not copy
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
453 del user[k]
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
454 else:
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
455 user[k] = _merge(user[k], v, filter_comments)
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
456 else:
8
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
457 user[k] = v
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
458 _filter_deletions(user)
8
7090c295c940 Two differend tree merge function implementations: not yet finished
Franz Glasner <hg@dom66.de>
parents: 5
diff changeset
459 return user
10
58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 9
diff changeset
460
58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 9
diff changeset
461
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
462 def _merge(user, default, filter_comments):
275
e2fd8fea1a4c Docu: more local lookup
Franz Glasner <fzglas.hg@dom66.de>
parents: 274
diff changeset
463 """Recursion helper for :func:`.merge`
110
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
464
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
465 """
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
466 if isinstance(user, dict) and isinstance(default, dict):
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
467 for k, v in default.items():
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
468 if filter_comments and _is_comment(k):
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
469 continue
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
470 if k in user:
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
471 if user[k] == DEL_VALUE:
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
472 # do not copy
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
473 del user[k]
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
474 else:
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
475 user[k] = _merge(user[k], v, filter_comments)
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
476 else:
110
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
477 user[k] = v
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
478 return user
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
479
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
480
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
481 def safe_merge(user, default, filter_comments=True):
275
e2fd8fea1a4c Docu: more local lookup
Franz Glasner <fzglas.hg@dom66.de>
parents: 274
diff changeset
482 """A more safe version of :func:`.merge` that makes deep copies of
13
24ba462b9b4b Return the `default' argument when the given `user' argument is `None' and when it is the first call on merge
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 11
diff changeset
483 the returned container objects.
24ba462b9b4b Return the `default' argument when the given `user' argument is `None' and when it is the first call on merge
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 11
diff changeset
484
275
e2fd8fea1a4c Docu: more local lookup
Franz Glasner <fzglas.hg@dom66.de>
parents: 274
diff changeset
485 Contrary to :func:`.merge` no given argument is ever changed
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
486 inplace. Every object from `default` is decoupled from the result
165
6ca90e80f4f4 FIX: Docu: wording
Franz Glasner <fzglas.hg@dom66.de>
parents: 163
diff changeset
487 -- so changing the `default` configuration later does not propagate
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
488 into a merged configuration later.
111
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
489
10
58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 9
diff changeset
490 """
110
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions
Franz Glasner <hg@dom66.de>
parents: 108
diff changeset
491 if user is None:
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
492 if filter_comments:
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
493 _filter_comments(default)
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
494 _filter_deletions(default)
111
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
495 return copy.deepcopy(default)
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
496 user = copy.deepcopy(user)
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
497 if filter_comments:
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
498 _filter_comments(user)
10
58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 9
diff changeset
499 if isinstance(user, dict) and isinstance(default, dict):
58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 9
diff changeset
500 for k, v in default.items():
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
501 if filter_comments and _is_comment(k):
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
502 continue
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
503 if k in user:
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
504 if user[k] == DEL_VALUE:
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
505 # do not copy
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
506 del user[k]
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
507 else:
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
508 user[k] = _safe_merge(user[k], v, filter_comments)
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
509 else:
111
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
510 user[k] = copy.deepcopy(v)
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
511 _filter_deletions(user)
10
58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 9
diff changeset
512 return user
111
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
513
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
514
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
515 def _safe_merge(user, default, filter_comments):
274
90bbade12d8e Docu: use local lookup firstly
Franz Glasner <fzglas.hg@dom66.de>
parents: 273
diff changeset
516 """Recursion helper for :func:`safe_merge`
111
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
517
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
518 """
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
519 if isinstance(user, dict) and isinstance(default, dict):
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
520 for k, v in default.items():
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
521 if filter_comments and _is_comment(k):
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
522 continue
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
523 if k in user:
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
524 if user[k] == DEL_VALUE:
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
525 # do not copy
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
526 del user[k]
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
527 else:
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
528 user[k] = _safe_merge(user[k], v, filter_comments)
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
529 else:
111
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
530 user[k] = copy.deepcopy(v)
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging.
Franz Glasner <hg@dom66.de>
parents: 110
diff changeset
531 return user
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
532
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
533
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
534 def _filter_comments(d):
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
535 """Recursively filter comments keys in the dict `d`.
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
536
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
537 Comment keys are keys that start with any of the items in
274
90bbade12d8e Docu: use local lookup firstly
Franz Glasner <fzglas.hg@dom66.de>
parents: 273
diff changeset
538 :data:`.COMMENTS`.
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
539
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
540 """
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
541 if not isinstance(d, dict):
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
542 return
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
543 # use a copy of the keys because we change `d` while iterating
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
544 for k in list(d.keys()):
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
545 if _is_comment(k):
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
546 del d[k]
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
547 else:
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
548 if isinstance(d[k], dict):
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
549 _filter_comments(d[k])
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
550
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
551
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
552 def _is_comment(k):
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
553 for i in COMMENTS:
256
7e26d31f52de FIX: Allow non-text keys when merging: handle .startswith() errors gracefully
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 253
diff changeset
554 try:
7e26d31f52de FIX: Allow non-text keys when merging: handle .startswith() errors gracefully
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 253
diff changeset
555 if k.startswith(i):
7e26d31f52de FIX: Allow non-text keys when merging: handle .startswith() errors gracefully
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 253
diff changeset
556 return True
7e26d31f52de FIX: Allow non-text keys when merging: handle .startswith() errors gracefully
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 253
diff changeset
557 except AttributeError:
7e26d31f52de FIX: Allow non-text keys when merging: handle .startswith() errors gracefully
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 253
diff changeset
558 # non-string key
7e26d31f52de FIX: Allow non-text keys when merging: handle .startswith() errors gracefully
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 253
diff changeset
559 return False
144
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
560 return False
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
561
7e6ec99d5ff5 Allow comments as keys and filter them by default
Franz Glasner <hg@dom66.de>
parents: 141
diff changeset
562
276
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
563 def _filter_deletions(d):
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
564 """Recursively filter deletions in the dict `d`.
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
565
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
566 Deletions have values that equal :data:`.DEL_VALUE`.
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
567
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
568 """
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
569 if not isinstance(d, dict):
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
570 return
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
571 # use a copy of the items because we change `d` while iterating
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
572 for k, v in list(d.items()):
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
573 if v == DEL_VALUE:
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
574 del d[k]
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
575 else:
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
576 if isinstance(d[k], dict):
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
577 _filter_deletions(d[k])
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
578
af371f9c016d Allow deletion of key-value pairs when merging is done.
Franz Glasner <fzglas.hg@dom66.de>
parents: 275
diff changeset
579
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
580 #
178
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions:
Franz Glasner <fzglas.hg@dom66.de>
parents: 177
diff changeset
581 # Init loader defaults: mode->loader and extension->mode
139
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles
Franz Glasner <hg@dom66.de>
parents: 138
diff changeset
582 #
189
0d0980ed74cc Rename the "_mode_loaders" variable to "mode_loaders" and make is therefore a "public" item
Franz Glasner <fzglas.hg@dom66.de>
parents: 188
diff changeset
583 mode_loaders.update(DEFAULT_MODE_LOADERS)
181
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
584 for _pattern, _mode in DEFAULT_ASSOC:
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search
Franz Glasner <fzglas.hg@dom66.de>
parents: 180
diff changeset
585 set_assoc(_pattern, _mode)