Python/libs/ConfigMix: configmix/__init_

annotate configmix/init.py @ 267:d715c10f2930

Allow an alternative to the "-*-ignore-*-" mode line: "-*- ignore -*-" (with spaces)

author	Franz Glasner <fzglas.hg@dom66.de>
date	Thu, 10 Sep 2020 02:01:28 +0200
parents	46571485b7d4
children	1484f6c0223a

rev	line source
0 53ea2bc254e7 Begin a package to abstract some of the important configuration handling stuff. Franz Glasner <hg@dom66.de> parents: diff changeset	1 # -- coding: utf-8 --
82 218807d7d883 Remove header markup from the Python files and put them into the doc .rst files Franz Glasner <hg@dom66.de> parents: 79 diff changeset	2 """A library for helping with configuration files.
4 f76d85ccc5b9 Switch to the "New BSD License" Franz Glasner <f.glasner@feldmann-mg.com> parents: 0 diff changeset	3
73 6216c561532a Put a Copyright and License notice into the package's __init__.py Franz Glasner <hg@dom66.de> parents: 69 diff changeset	4 :Author: Franz Glasner
237 13711ba8e81e Adjust copyright year to 2020 Franz Glasner <fzglas.hg@dom66.de> parents: 230 diff changeset	5 :Copyright: (c) 2015–2020, Franz Glasner.
73 6216c561532a Put a Copyright and License notice into the package's __init__.py Franz Glasner <hg@dom66.de> parents: 69 diff changeset	6 All rights reserved.
78 3a8f712d27df Put copyright and license into the README.txt Franz Glasner <hg@dom66.de> parents: 73 diff changeset	7 :License: 3-clause BSD License.
73 6216c561532a Put a Copyright and License notice into the package's __init__.py Franz Glasner <hg@dom66.de> parents: 69 diff changeset	8 See LICENSE.txt for details.
174 e2505f524ab9 Use the "@(#)" sigil in the package documentation header Franz Glasner <f.glasner@feldmann-mg.com> parents: 173 diff changeset	9 :ID: @(#) $Header$
4 f76d85ccc5b9 Switch to the "New BSD License" Franz Glasner <f.glasner@feldmann-mg.com> parents: 0 diff changeset	10
f76d85ccc5b9 Switch to the "New BSD License" Franz Glasner <f.glasner@feldmann-mg.com> parents: 0 diff changeset	11 """
0 53ea2bc254e7 Begin a package to abstract some of the important configuration handling stuff. Franz Glasner <hg@dom66.de> parents: diff changeset	12
53ea2bc254e7 Begin a package to abstract some of the important configuration handling stuff. Franz Glasner <hg@dom66.de> parents: diff changeset	13 from __future__ import division, print_function, absolute_import
53ea2bc254e7 Begin a package to abstract some of the important configuration handling stuff. Franz Glasner <hg@dom66.de> parents: diff changeset	14
53ea2bc254e7 Begin a package to abstract some of the important configuration handling stuff. Franz Glasner <hg@dom66.de> parents: diff changeset	15
262 f62764292b9d +++++ v0.9 Franz Glasner <fzglas.hg@dom66.de> parents: 257 diff changeset	16 __version__ = "0.9"
0 53ea2bc254e7 Begin a package to abstract some of the important configuration handling stuff. Franz Glasner <hg@dom66.de> parents: diff changeset	17
163 ff03cdf36139 The README.txt should be in the most important parts readable without keyword substitutions. Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	18 __revision__ = "\|VCSRevision\|"
ff03cdf36139 The README.txt should be in the most important parts readable without keyword substitutions. Franz Glasner <fzglas.hg@dom66.de> parents: 156 diff changeset	19 __date__ = "\|VCSJustDate\|"
52 6c7f90dbce98 Adjusted the Copyright and change the RCS keywords in accordance with "kwarchive" Franz Glasner <hg@dom66.de> parents: 45 diff changeset	20
177 6dde1e344ae8 Style: put "__all__" into the meta-variables section as recommended by PEP Franz Glasner <f.glasner@feldmann-mg.com> parents: 175 diff changeset	21 __all__ = ["load", "safe_load",
181 7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search Franz Glasner <fzglas.hg@dom66.de> parents: 180 diff changeset	22 "set_assoc", "get_assoc", "clear_assoc",
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search Franz Glasner <fzglas.hg@dom66.de> parents: 180 diff changeset	23 "get_default_assoc",
180 e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	24 "Configuration",
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	25 "try_determine_filemode"]
177 6dde1e344ae8 Style: put "__all__" into the meta-variables section as recommended by PEP Franz Glasner <f.glasner@feldmann-mg.com> parents: 175 diff changeset	26
0 53ea2bc254e7 Begin a package to abstract some of the important configuration handling stuff. Franz Glasner <hg@dom66.de> parents: diff changeset	27
171 1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now. Franz Glasner <f.glasner@feldmann-mg.com> parents: 170 diff changeset	28 import fnmatch
10 58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers Franz Glasner <f.glasner@feldmann-mg.com> parents: 9 diff changeset	29 import copy
180 e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	30 import io
266 46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	31 import os
180 e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	32 import re
10 58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers Franz Glasner <f.glasner@feldmann-mg.com> parents: 9 diff changeset	33
166 b5ce9a8461bf Use the filesystem encoding explicitely where appropriate. Franz Glasner <fzglas.hg@dom66.de> parents: 165 diff changeset	34 from .compat import u, u2fs
22 6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles Franz Glasner <f.glasner@feldmann-mg.com> parents: 13 diff changeset	35 from .config import Configuration
10 58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers Franz Glasner <f.glasner@feldmann-mg.com> parents: 9 diff changeset	36
22 6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles Franz Glasner <f.glasner@feldmann-mg.com> parents: 13 diff changeset	37
207 b3b5ed34d180 Handle most flake8 errors and warnings. Franz Glasner <fzglas.hg@dom66.de> parents: 206 diff changeset	38 COMMENTS = [
b3b5ed34d180 Handle most flake8 errors and warnings. Franz Glasner <fzglas.hg@dom66.de> parents: 206 diff changeset	39 u("__comment"),
b3b5ed34d180 Handle most flake8 errors and warnings. Franz Glasner <fzglas.hg@dom66.de> parents: 206 diff changeset	40 u("__doc"),
144 7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	41 ]
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	42 """Prefixes for comment configuration keys that are to be handled as
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	43 comments
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	44
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	45 """
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	46
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	47
172 8138d56d7cd3 ".load" and ".safe_load" get a keyword parameter "defaults" that allows the provision of a configuration dictionary with default settings Franz Glasner <f.glasner@feldmann-mg.com> parents: 171 diff changeset	48 def load(files, *kwargs):
22 6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles Franz Glasner <f.glasner@feldmann-mg.com> parents: 13 diff changeset	49 """Load the given configuration files, merge them in the given order
108 2196362c0467 Enhance documentation of "configmix.load()" Franz Glasner <hg@dom66.de> parents: 107 diff changeset	50 and return the resulting configuration dictionary.
2196362c0467 Enhance documentation of "configmix.load()" Franz Glasner <hg@dom66.de> parents: 107 diff changeset	51
2196362c0467 Enhance documentation of "configmix.load()" Franz Glasner <hg@dom66.de> parents: 107 diff changeset	52 :param files: the filenames of the configuration files to read and merge
173 b3ba2b0265b5 Docu: Explicitely tag "defaults" as keyword argument Franz Glasner <f.glasner@feldmann-mg.com> parents: 172 diff changeset	53 :keyword defaults: optional configuration dictionary with some default
b3ba2b0265b5 Docu: Explicitely tag "defaults" as keyword argument Franz Glasner <f.glasner@feldmann-mg.com> parents: 172 diff changeset	54 settings where the settings from `files` are merged
b3ba2b0265b5 Docu: Explicitely tag "defaults" as keyword argument Franz Glasner <f.glasner@feldmann-mg.com> parents: 172 diff changeset	55 into
214 a35b0ca8b81f FIX: Docu: Sphinx markup Franz Glasner <fzglas.hg@dom66.de> parents: 211 diff changeset	56 :type defaults: dict-alike or None
221 6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary Franz Glasner <fzglas.hg@dom66.de> parents: 220 diff changeset	57 :keyword extras: optional configuration dictionary that will applied
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary Franz Glasner <fzglas.hg@dom66.de> parents: 220 diff changeset	58 last
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary Franz Glasner <fzglas.hg@dom66.de> parents: 220 diff changeset	59
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary Franz Glasner <fzglas.hg@dom66.de> parents: 220 diff changeset	60 Use this for example to overwrite configuration file
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary Franz Glasner <fzglas.hg@dom66.de> parents: 220 diff changeset	61 settings from commandline arguments.
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary Franz Glasner <fzglas.hg@dom66.de> parents: 220 diff changeset	62 :type extras: dict-alike or None
108 2196362c0467 Enhance documentation of "configmix.load()" Franz Glasner <hg@dom66.de> parents: 107 diff changeset	63 :returns: the configuration
2196362c0467 Enhance documentation of "configmix.load()" Franz Glasner <hg@dom66.de> parents: 107 diff changeset	64 :rtype: ~configmix.config.Configuration
22 6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles Franz Glasner <f.glasner@feldmann-mg.com> parents: 13 diff changeset	65
6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles Franz Glasner <f.glasner@feldmann-mg.com> parents: 13 diff changeset	66 """
172 8138d56d7cd3 ".load" and ".safe_load" get a keyword parameter "defaults" that allows the provision of a configuration dictionary with default settings Franz Glasner <f.glasner@feldmann-mg.com> parents: 171 diff changeset	67 defaults = kwargs.get("defaults")
221 6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary Franz Glasner <fzglas.hg@dom66.de> parents: 220 diff changeset	68 extras = kwargs.get("extras")
217 b869e792310e FIX: Extra merged and unsafe merges in configmix.safe_load Franz Glasner <fzglas.hg@dom66.de> parents: 216 diff changeset	69 if defaults is None:
220 2034da70f8fd Simplify the implementation of configmix.load() and .safe_load(): Franz Glasner <fzglas.hg@dom66.de> parents: 217 diff changeset	70 ex = Configuration()
22 6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles Franz Glasner <f.glasner@feldmann-mg.com> parents: 13 diff changeset	71 else:
220 2034da70f8fd Simplify the implementation of configmix.load() and .safe_load(): Franz Glasner <fzglas.hg@dom66.de> parents: 217 diff changeset	72 ex = merge(None, Configuration(defaults))
2034da70f8fd Simplify the implementation of configmix.load() and .safe_load(): Franz Glasner <fzglas.hg@dom66.de> parents: 217 diff changeset	73 for f in files:
266 46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	74 if f.startswith("<dir>"):
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	75 for f2 in _get_configuration_files_from_dir(f[5:]):
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	76 nx = _load_cfg_from_file(f2, ignore_unknown=True)
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	77 if nx is not None:
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	78 ex = merge(nx, ex)
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	79 else:
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	80 nx = _load_cfg_from_file(f)
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	81 if nx is not None:
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	82 ex = merge(nx, ex)
221 6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary Franz Glasner <fzglas.hg@dom66.de> parents: 220 diff changeset	83 if extras:
6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary Franz Glasner <fzglas.hg@dom66.de> parents: 220 diff changeset	84 ex = merge(Configuration(extras), ex)
217 b869e792310e FIX: Extra merged and unsafe merges in configmix.safe_load Franz Glasner <fzglas.hg@dom66.de> parents: 216 diff changeset	85 return Configuration(ex)
22 6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles Franz Glasner <f.glasner@feldmann-mg.com> parents: 13 diff changeset	86
6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles Franz Glasner <f.glasner@feldmann-mg.com> parents: 13 diff changeset	87
172 8138d56d7cd3 ".load" and ".safe_load" get a keyword parameter "defaults" that allows the provision of a configuration dictionary with default settings Franz Glasner <f.glasner@feldmann-mg.com> parents: 171 diff changeset	88 def safe_load(files, *kwargs):
112 c50ad93eb5dc Implemented a "safe_load()" to load with safe merging Franz Glasner <hg@dom66.de> parents: 111 diff changeset	89 """Analogous to :func:`load` but do merging with :func:`safe_merge`
c50ad93eb5dc Implemented a "safe_load()" to load with safe merging Franz Glasner <hg@dom66.de> parents: 111 diff changeset	90 instead of :func:`merge`
c50ad93eb5dc Implemented a "safe_load()" to load with safe merging Franz Glasner <hg@dom66.de> parents: 111 diff changeset	91
c50ad93eb5dc Implemented a "safe_load()" to load with safe merging Franz Glasner <hg@dom66.de> parents: 111 diff changeset	92 """
172 8138d56d7cd3 ".load" and ".safe_load" get a keyword parameter "defaults" that allows the provision of a configuration dictionary with default settings Franz Glasner <f.glasner@feldmann-mg.com> parents: 171 diff changeset	93 defaults = kwargs.get("defaults")
221 6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary Franz Glasner <fzglas.hg@dom66.de> parents: 220 diff changeset	94 extras = kwargs.get("extras")
217 b869e792310e FIX: Extra merged and unsafe merges in configmix.safe_load Franz Glasner <fzglas.hg@dom66.de> parents: 216 diff changeset	95 if defaults is None:
220 2034da70f8fd Simplify the implementation of configmix.load() and .safe_load(): Franz Glasner <fzglas.hg@dom66.de> parents: 217 diff changeset	96 ex = Configuration()
112 c50ad93eb5dc Implemented a "safe_load()" to load with safe merging Franz Glasner <hg@dom66.de> parents: 111 diff changeset	97 else:
220 2034da70f8fd Simplify the implementation of configmix.load() and .safe_load(): Franz Glasner <fzglas.hg@dom66.de> parents: 217 diff changeset	98 ex = safe_merge(None, Configuration(defaults))
2034da70f8fd Simplify the implementation of configmix.load() and .safe_load(): Franz Glasner <fzglas.hg@dom66.de> parents: 217 diff changeset	99 for f in files:
266 46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	100 if f.startswith("<dir>"):
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	101 for f2 in _get_configuration_files_from_dir(f[5:]):
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	102 nx = _load_cfg_from_file(f2, ignore_unknown=True)
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	103 if nx is not None:
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	104 ex = safe_merge(nx, ex)
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	105 else:
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	106 nx = _load_cfg_from_file(f)
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	107 if nx is not None:
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	108 ex = safe_merge(nx, ex)
221 6f0f39a9a46f configmix.load() and .safe_load() got a new keyword argument "extras" to be merged in as last configuration dictionary Franz Glasner <fzglas.hg@dom66.de> parents: 220 diff changeset	109 if extras:
223 90dd0d04b926 +++++ v0.7.1 Franz Glasner <fzglas.hg@dom66.de> parents: 221 diff changeset	110 ex = safe_merge(Configuration(extras), ex)
217 b869e792310e FIX: Extra merged and unsafe merges in configmix.safe_load Franz Glasner <fzglas.hg@dom66.de> parents: 216 diff changeset	111 return Configuration(ex)
112 c50ad93eb5dc Implemented a "safe_load()" to load with safe merging Franz Glasner <hg@dom66.de> parents: 111 diff changeset	112
c50ad93eb5dc Implemented a "safe_load()" to load with safe merging Franz Glasner <hg@dom66.de> parents: 111 diff changeset	113
266 46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	114 def _get_configuration_files_from_dir(root):
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	115 """Returns the sorted list of files within directory `root`
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	116
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	117 """
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	118 files = []
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	119
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	120 if not os.path.isdir(root):
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	121 return files
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	122
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	123 dirfiles = os.listdir(root)
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	124 dirfiles.sort()
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	125 for f in dirfiles:
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	126 path = os.path.join(root, f)
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	127 if os.path.isdir(path):
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	128 # XXX TBD Recurse??? depth-first???
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	129 continue
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	130 files.append(path)
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	131 return files
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	132
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	133
138 b883f4ef1967 Indirectly map extensions to configuration file styles Franz Glasner <hg@dom66.de> parents: 122 diff changeset	134 def _load_yaml(filename):
b883f4ef1967 Indirectly map extensions to configuration file styles Franz Glasner <hg@dom66.de> parents: 122 diff changeset	135 from . import yaml
166 b5ce9a8461bf Use the filesystem encoding explicitely where appropriate. Franz Glasner <fzglas.hg@dom66.de> parents: 165 diff changeset	136 with open(u2fs(filename), "rb") as yf:
138 b883f4ef1967 Indirectly map extensions to configuration file styles Franz Glasner <hg@dom66.de> parents: 122 diff changeset	137 return yaml.safe_load(yf)
b883f4ef1967 Indirectly map extensions to configuration file styles Franz Glasner <hg@dom66.de> parents: 122 diff changeset	138
b883f4ef1967 Indirectly map extensions to configuration file styles Franz Glasner <hg@dom66.de> parents: 122 diff changeset	139
b883f4ef1967 Indirectly map extensions to configuration file styles Franz Glasner <hg@dom66.de> parents: 122 diff changeset	140 def _load_json(filename):
b883f4ef1967 Indirectly map extensions to configuration file styles Franz Glasner <hg@dom66.de> parents: 122 diff changeset	141 from . import json
b883f4ef1967 Indirectly map extensions to configuration file styles Franz Glasner <hg@dom66.de> parents: 122 diff changeset	142 return json.load(filename)
b883f4ef1967 Indirectly map extensions to configuration file styles Franz Glasner <hg@dom66.de> parents: 122 diff changeset	143
b883f4ef1967 Indirectly map extensions to configuration file styles Franz Glasner <hg@dom66.de> parents: 122 diff changeset	144
b883f4ef1967 Indirectly map extensions to configuration file styles Franz Glasner <hg@dom66.de> parents: 122 diff changeset	145 def _load_py(filename):
b883f4ef1967 Indirectly map extensions to configuration file styles Franz Glasner <hg@dom66.de> parents: 122 diff changeset	146 from . import py
b883f4ef1967 Indirectly map extensions to configuration file styles Franz Glasner <hg@dom66.de> parents: 122 diff changeset	147 return py.load(filename)
b883f4ef1967 Indirectly map extensions to configuration file styles Franz Glasner <hg@dom66.de> parents: 122 diff changeset	148
b883f4ef1967 Indirectly map extensions to configuration file styles Franz Glasner <hg@dom66.de> parents: 122 diff changeset	149
b883f4ef1967 Indirectly map extensions to configuration file styles Franz Glasner <hg@dom66.de> parents: 122 diff changeset	150 def _load_ini(filename):
b883f4ef1967 Indirectly map extensions to configuration file styles Franz Glasner <hg@dom66.de> parents: 122 diff changeset	151 from . import ini
b883f4ef1967 Indirectly map extensions to configuration file styles Franz Glasner <hg@dom66.de> parents: 122 diff changeset	152 return ini.load(filename)
b883f4ef1967 Indirectly map extensions to configuration file styles Franz Glasner <hg@dom66.de> parents: 122 diff changeset	153
b883f4ef1967 Indirectly map extensions to configuration file styles Franz Glasner <hg@dom66.de> parents: 122 diff changeset	154
195 28e6c1413947 Added support for TOML style configuration files Franz Glasner <fzglas.hg@dom66.de> parents: 191 diff changeset	155 def _load_toml(filename):
28e6c1413947 Added support for TOML style configuration files Franz Glasner <fzglas.hg@dom66.de> parents: 191 diff changeset	156 from . import toml
28e6c1413947 Added support for TOML style configuration files Franz Glasner <fzglas.hg@dom66.de> parents: 191 diff changeset	157 return toml.load(filename)
28e6c1413947 Added support for TOML style configuration files Franz Glasner <fzglas.hg@dom66.de> parents: 191 diff changeset	158
28e6c1413947 Added support for TOML style configuration files Franz Glasner <fzglas.hg@dom66.de> parents: 191 diff changeset	159
227 f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file Franz Glasner <fzglas.hg@dom66.de> parents: 226 diff changeset	160 def _load_ignore(filename):
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file Franz Glasner <fzglas.hg@dom66.de> parents: 226 diff changeset	161 """A loader that returns `None` just to ignore `filename`"""
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file Franz Glasner <fzglas.hg@dom66.de> parents: 226 diff changeset	162 return None
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file Franz Glasner <fzglas.hg@dom66.de> parents: 226 diff changeset	163
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file Franz Glasner <fzglas.hg@dom66.de> parents: 226 diff changeset	164
180 e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	165 EMACS_MODELINE = re.compile(r"-\-(.?)-\*-")
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	166 EMACS_MODE = re.compile(r"(?:\A\s\|;\s)mode[:=]\s*([-_.a-zA-Z0-9]+)")
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	167
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	168
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	169 def try_determine_filemode(filename):
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	170 """Try to determine an explicitely given filemode from an Emacs-compatible
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	171 mode declaration (e.g. ``mode=python``).
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	172
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	173 :param str filename:
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	174 :return: the found mode string or `None`
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	175 :rtype: str or None
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	176
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	177 Only the first two lines are searched for.
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	178
206 5064e3a2e54a Doc: "configmix.try_determine_filemode()" got additional documentation Franz Glasner <fzglas.hg@dom66.de> parents: 197 diff changeset	179 Conveniently to be used in calls to :func:`~.set_assoc` to determine
5064e3a2e54a Doc: "configmix.try_determine_filemode()" got additional documentation Franz Glasner <fzglas.hg@dom66.de> parents: 197 diff changeset	180 the file-mode by content instead of filename extension.
5064e3a2e54a Doc: "configmix.try_determine_filemode()" got additional documentation Franz Glasner <fzglas.hg@dom66.de> parents: 197 diff changeset	181
180 e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	182 """
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	183 with io.open(filename, encoding="ascii", errors="replace") as f:
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	184 idx = 0
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	185 for l in f:
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	186 idx += 1
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	187 mo = EMACS_MODELINE.search(l)
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	188 if mo:
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	189 mo = EMACS_MODE.search(mo.group(1))
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	190 if mo:
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	191 return mo.group(1)
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	192 if idx >= 2:
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	193 break
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	194 return None
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	195
e87fa5bd68e7 Implemented "try_determine_filemode()" to determine a file-mode from an Emacs-compatible declaration Franz Glasner <fzglas.hg@dom66.de> parents: 179 diff changeset	196
178 eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	197 DEFAULT_MODE_LOADERS = {
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	198 "python": _load_py,
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	199 "yaml": _load_yaml,
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	200 "conf": _load_ini,
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	201 "conf-windows": _load_ini,
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	202 "ini": _load_ini,
195 28e6c1413947 Added support for TOML style configuration files Franz Glasner <fzglas.hg@dom66.de> parents: 191 diff changeset	203 "toml": _load_toml,
197 7865f28038a4 Add the "conf-toml" mode mapping to the TOML loader Franz Glasner <fzglas.hg@dom66.de> parents: 195 diff changeset	204 "conf-toml": _load_toml,
178 eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	205 "javascript": _load_json,
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	206 "json": _load_json,
228 b2c75efad9e4 Renamed the 'ignore' loader key to '--ignore--' Franz Glasner <fzglas.hg@dom66.de> parents: 227 diff changeset	207 "--ignore--": _load_ignore,
267 d715c10f2930 Allow an alternative to the "--ignore--" mode line: "-- ignore --" (with spaces) Franz Glasner <fzglas.hg@dom66.de> parents: 266 diff changeset	208 "-- ignore --": _load_ignore,
178 eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	209 }
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	210 """Default associations between file modes and loader functions"""
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	211
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	212
181 7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search Franz Glasner <fzglas.hg@dom66.de> parents: 180 diff changeset	213 DEFAULT_ASSOC = [
178 eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	214 ("*.yml", "yaml"),
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	215 ("*.yaml", "yaml"),
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	216 ("*.json", "json"),
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	217 ("*.py", "python"),
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	218 ("*.ini", "conf"),
195 28e6c1413947 Added support for TOML style configuration files Franz Glasner <fzglas.hg@dom66.de> parents: 191 diff changeset	219 ("*.toml", "toml"),
171 1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now. Franz Glasner <f.glasner@feldmann-mg.com> parents: 170 diff changeset	220 ]
178 eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	221 """The builtin default associations of filename extensions with
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	222 file modes -- in that order.
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	223
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	224 The "mode" part may be a string or a callable with a filename
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	225 parameter that returns the mode string for the file or `None` if it
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	226 can not determined.
171 1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now. Franz Glasner <f.glasner@feldmann-mg.com> parents: 170 diff changeset	227
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now. Franz Glasner <f.glasner@feldmann-mg.com> parents: 170 diff changeset	228 """
139 c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles Franz Glasner <hg@dom66.de> parents: 138 diff changeset	229
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles Franz Glasner <hg@dom66.de> parents: 138 diff changeset	230
188 17b938ccecb8 Rename the DEFAULT_LOADER marker to USE_DEFAULT_ASSOC Franz Glasner <fzglas.hg@dom66.de> parents: 187 diff changeset	231 USE_DEFAULT_ASSOC = object()
181 7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search Franz Glasner <fzglas.hg@dom66.de> parents: 180 diff changeset	232 """Marker for the default association for an extension.
139 c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles Franz Glasner <hg@dom66.de> parents: 138 diff changeset	233
181 7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search Franz Glasner <fzglas.hg@dom66.de> parents: 180 diff changeset	234 To be used in :func:`set_assoc`.
139 c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles Franz Glasner <hg@dom66.de> parents: 138 diff changeset	235 """
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles Franz Glasner <hg@dom66.de> parents: 138 diff changeset	236
181 7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search Franz Glasner <fzglas.hg@dom66.de> parents: 180 diff changeset	237
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search Franz Glasner <fzglas.hg@dom66.de> parents: 180 diff changeset	238 def get_default_assoc(pattern):
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search Franz Glasner <fzglas.hg@dom66.de> parents: 180 diff changeset	239 """Return the default file-mode association for the :mod:`fnmatch`
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search Franz Glasner <fzglas.hg@dom66.de> parents: 180 diff changeset	240 pattern `pattern`.
171 1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now. Franz Glasner <f.glasner@feldmann-mg.com> parents: 170 diff changeset	241
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now. Franz Glasner <f.glasner@feldmann-mg.com> parents: 170 diff changeset	242 :raises: :class:`KeyError` if the `pattern` is not found.
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now. Franz Glasner <f.glasner@feldmann-mg.com> parents: 170 diff changeset	243
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now. Franz Glasner <f.glasner@feldmann-mg.com> parents: 170 diff changeset	244 """
181 7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search Franz Glasner <fzglas.hg@dom66.de> parents: 180 diff changeset	245 for pat, fmode in DEFAULT_ASSOC:
171 1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now. Franz Glasner <f.glasner@feldmann-mg.com> parents: 170 diff changeset	246 if pattern == pat:
181 7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search Franz Glasner <fzglas.hg@dom66.de> parents: 180 diff changeset	247 return fmode
171 1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now. Franz Glasner <f.glasner@feldmann-mg.com> parents: 170 diff changeset	248 else:
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now. Franz Glasner <f.glasner@feldmann-mg.com> parents: 170 diff changeset	249 raise KeyError("No loader for pattern %r" % pattern)
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now. Franz Glasner <f.glasner@feldmann-mg.com> parents: 170 diff changeset	250
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now. Franz Glasner <f.glasner@feldmann-mg.com> parents: 170 diff changeset	251
189 0d0980ed74cc Rename the "_mode_loaders" variable to "mode_loaders" and make is therefore a "public" item Franz Glasner <fzglas.hg@dom66.de> parents: 188 diff changeset	252 mode_loaders = {}
178 eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	253 """All configured associations between file modes and loader functions.
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	254
191 8af600d0e5c0 FIX: Docu: added missing backtick Franz Glasner <fzglas.hg@dom66.de> parents: 189 diff changeset	255 See :data:`DEFAULT_MODE_LOADERS`.
178 eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	256
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	257 """
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	258
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	259 _extensions = []
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	260 """All configured assiciations of filename extensions with file modes.
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	261
181 7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search Franz Glasner <fzglas.hg@dom66.de> parents: 180 diff changeset	262 See :data:`DEFAULT_ASSOC`
178 eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	263
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	264 """
139 c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles Franz Glasner <hg@dom66.de> parents: 138 diff changeset	265
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles Franz Glasner <hg@dom66.de> parents: 138 diff changeset	266
181 7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search Franz Glasner <fzglas.hg@dom66.de> parents: 180 diff changeset	267 def clear_assoc():
139 c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles Franz Glasner <hg@dom66.de> parents: 138 diff changeset	268 """Remove all configured loader associations.
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles Franz Glasner <hg@dom66.de> parents: 138 diff changeset	269
187 d2eb83720ad8 FIX: Docu: names of predefined constants Franz Glasner <fzglas.hg@dom66.de> parents: 186 diff changeset	270 The :data:`.DEFAULT_ASSOC` are not changed.
139 c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles Franz Glasner <hg@dom66.de> parents: 138 diff changeset	271
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles Franz Glasner <hg@dom66.de> parents: 138 diff changeset	272 """
181 7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search Franz Glasner <fzglas.hg@dom66.de> parents: 180 diff changeset	273 del _extensions[:]
139 c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles Franz Glasner <hg@dom66.de> parents: 138 diff changeset	274
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles Franz Glasner <hg@dom66.de> parents: 138 diff changeset	275
181 7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search Franz Glasner <fzglas.hg@dom66.de> parents: 180 diff changeset	276 def get_assoc(pattern):
171 1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now. Franz Glasner <f.glasner@feldmann-mg.com> parents: 170 diff changeset	277 """Return the default loader for the :mod:`fnmatch` pattern `pattern`.
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now. Franz Glasner <f.glasner@feldmann-mg.com> parents: 170 diff changeset	278
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now. Franz Glasner <f.glasner@feldmann-mg.com> parents: 170 diff changeset	279 :raises: :class:`KeyError` if the `pattern` is not found.
139 c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles Franz Glasner <hg@dom66.de> parents: 138 diff changeset	280
171 1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now. Franz Glasner <f.glasner@feldmann-mg.com> parents: 170 diff changeset	281 """
181 7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search Franz Glasner <fzglas.hg@dom66.de> parents: 180 diff changeset	282 for pat, fmode in _extensions:
171 1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now. Franz Glasner <f.glasner@feldmann-mg.com> parents: 170 diff changeset	283 if pattern == pat:
181 7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search Franz Glasner <fzglas.hg@dom66.de> parents: 180 diff changeset	284 return fmode
171 1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now. Franz Glasner <f.glasner@feldmann-mg.com> parents: 170 diff changeset	285 else:
181 7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search Franz Glasner <fzglas.hg@dom66.de> parents: 180 diff changeset	286 raise KeyError("No associated file-mode for pattern %r" % pattern)
171 1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now. Franz Glasner <f.glasner@feldmann-mg.com> parents: 170 diff changeset	287
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now. Franz Glasner <f.glasner@feldmann-mg.com> parents: 170 diff changeset	288
186 fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()" Franz Glasner <fzglas.hg@dom66.de> parents: 181 diff changeset	289 def set_assoc(fnpattern, mode, append=False):
179 62db35db8939 Docu: formatting Franz Glasner <f.glasner@feldmann-mg.com> parents: 178 diff changeset	290 """Associate a :mod:`fnmatch` style pattern `fnpattern` with a
62db35db8939 Docu: formatting Franz Glasner <f.glasner@feldmann-mg.com> parents: 178 diff changeset	291 file-mode `mode` that determines what will be called when
62db35db8939 Docu: formatting Franz Glasner <f.glasner@feldmann-mg.com> parents: 178 diff changeset	292 :func:`load` encounters a file argument that matches `fnpattern`.
171 1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now. Franz Glasner <f.glasner@feldmann-mg.com> parents: 170 diff changeset	293
179 62db35db8939 Docu: formatting Franz Glasner <f.glasner@feldmann-mg.com> parents: 178 diff changeset	294 :param str fnpattern: the :mod:`fnmatch` pattern to associate a loader
62db35db8939 Docu: formatting Franz Glasner <f.glasner@feldmann-mg.com> parents: 178 diff changeset	295 with
186 fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()" Franz Glasner <fzglas.hg@dom66.de> parents: 181 diff changeset	296 :param mode: a mode string or a callable that accepts a `filename`
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()" Franz Glasner <fzglas.hg@dom66.de> parents: 181 diff changeset	297 argument and returns a file-mode for the given file
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()" Franz Glasner <fzglas.hg@dom66.de> parents: 181 diff changeset	298 (or `None`)
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()" Franz Glasner <fzglas.hg@dom66.de> parents: 181 diff changeset	299 :type mode: str or callable
139 c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles Franz Glasner <hg@dom66.de> parents: 138 diff changeset	300
186 fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()" Franz Glasner <fzglas.hg@dom66.de> parents: 181 diff changeset	301 :keyword bool append: If `False` (which is the default) then this
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()" Franz Glasner <fzglas.hg@dom66.de> parents: 181 diff changeset	302 function inserts the given pattern at the head position of the
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()" Franz Glasner <fzglas.hg@dom66.de> parents: 181 diff changeset	303 currently defined associations, if `True` the pattern will be appended
171 1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now. Franz Glasner <f.glasner@feldmann-mg.com> parents: 170 diff changeset	304
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now. Franz Glasner <f.glasner@feldmann-mg.com> parents: 170 diff changeset	305 The OS specific case-sensitivity behaviour of
175 327032bb0f6b Docu: wording Franz Glasner <f.glasner@feldmann-mg.com> parents: 174 diff changeset	306 :func:`fnmatch.fnmatch` applies (i.e. :func:`os.path.normpath`
327032bb0f6b Docu: wording Franz Glasner <f.glasner@feldmann-mg.com> parents: 174 diff changeset	307 will be called for both arguments).
141 647782859ae1 An extra hint that filename extension comparisons for loader lookup are case-insensitive Franz Glasner <hg@dom66.de> parents: 139 diff changeset	308
188 17b938ccecb8 Rename the DEFAULT_LOADER marker to USE_DEFAULT_ASSOC Franz Glasner <fzglas.hg@dom66.de> parents: 187 diff changeset	309 If `loader` is :data:`.USE_DEFAULT_ASSOC` then the default association
187 d2eb83720ad8 FIX: Docu: names of predefined constants Franz Glasner <fzglas.hg@dom66.de> parents: 186 diff changeset	310 from :data:`.DEFAULT_ASSOC` will be used -- if any.
139 c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles Franz Glasner <hg@dom66.de> parents: 138 diff changeset	311
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles Franz Glasner <hg@dom66.de> parents: 138 diff changeset	312 """
188 17b938ccecb8 Rename the DEFAULT_LOADER marker to USE_DEFAULT_ASSOC Franz Glasner <fzglas.hg@dom66.de> parents: 187 diff changeset	313 if mode is USE_DEFAULT_ASSOC:
181 7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search Franz Glasner <fzglas.hg@dom66.de> parents: 180 diff changeset	314 for p, m in DEFAULT_ASSOC:
178 eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	315 if p == fnpattern:
186 fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()" Franz Glasner <fzglas.hg@dom66.de> parents: 181 diff changeset	316 if append:
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()" Franz Glasner <fzglas.hg@dom66.de> parents: 181 diff changeset	317 _extensions.append((fnpattern, m))
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()" Franz Glasner <fzglas.hg@dom66.de> parents: 181 diff changeset	318 else:
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()" Franz Glasner <fzglas.hg@dom66.de> parents: 181 diff changeset	319 _extensions.insert(0, (fnpattern, m))
171 1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now. Franz Glasner <f.glasner@feldmann-mg.com> parents: 170 diff changeset	320 break
1ff11462a5c1 The associations from filename extensions to parsers are "fnmatch" style patterns now. Franz Glasner <f.glasner@feldmann-mg.com> parents: 170 diff changeset	321 else:
178 eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	322 raise ValueError("no DEFAULT mode for pattern: %r" % fnpattern)
139 c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles Franz Glasner <hg@dom66.de> parents: 138 diff changeset	323 else:
186 fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()" Franz Glasner <fzglas.hg@dom66.de> parents: 181 diff changeset	324 if append:
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()" Franz Glasner <fzglas.hg@dom66.de> parents: 181 diff changeset	325 _extensions.append((fnpattern, mode))
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()" Franz Glasner <fzglas.hg@dom66.de> parents: 181 diff changeset	326 else:
fa101fb0cd7a Implement an "append" keyword to "configmix.set_assoc()" Franz Glasner <fzglas.hg@dom66.de> parents: 181 diff changeset	327 _extensions.insert(0, (fnpattern, mode))
138 b883f4ef1967 Indirectly map extensions to configuration file styles Franz Glasner <hg@dom66.de> parents: 122 diff changeset	328
b883f4ef1967 Indirectly map extensions to configuration file styles Franz Glasner <hg@dom66.de> parents: 122 diff changeset	329
266 46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	330 def _load_cfg_from_file(filename, ignore_unknown=False):
227 f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file Franz Glasner <fzglas.hg@dom66.de> parents: 226 diff changeset	331 """Determine the loader for file `filename` and return the loaded
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file Franz Glasner <fzglas.hg@dom66.de> parents: 226 diff changeset	332 configuration dict.
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file Franz Glasner <fzglas.hg@dom66.de> parents: 226 diff changeset	333
266 46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	334 If `ignore_unknown` is `True` then unknown extensions are ignored.
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	335 Otherwise a :exc:`ValueError` exception is raised.
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	336
227 f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file Franz Glasner <fzglas.hg@dom66.de> parents: 226 diff changeset	337 Can return `None` is the file should be ignored by the caller.
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file Franz Glasner <fzglas.hg@dom66.de> parents: 226 diff changeset	338
f5011eec3b6e Added a loader with key "ignore" that ignores the given configuration file Franz Glasner <fzglas.hg@dom66.de> parents: 226 diff changeset	339 """
178 eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	340 for p, m in _extensions:
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	341 if fnmatch.fnmatch(filename, p):
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	342 if callable(m):
eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	343 m = m(filename)
265 39c379fa8c65 FIX: Check the return value when calling only when the callable has been really called. Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	344 if m is None:
39c379fa8c65 FIX: Check the return value when calling only when the callable has been really called. Franz Glasner <fzglas.hg@dom66.de> parents: 262 diff changeset	345 continue
189 0d0980ed74cc Rename the "_mode_loaders" variable to "mode_loaders" and make is therefore a "public" item Franz Glasner <fzglas.hg@dom66.de> parents: 188 diff changeset	346 return mode_loaders[m](filename)
22 6a91db2c2469 A convenience function to load and merge a list of configuration files with different styles Franz Glasner <f.glasner@feldmann-mg.com> parents: 13 diff changeset	347 else:
266 46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	348 if ignore_unknown:
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	349 return None
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	350 else:
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	351 raise ValueError("Unknown configuration file type for filename "
46571485b7d4 Allow loading configuration files from directories when using the "<dir>" prefix in filenames. Franz Glasner <fzglas.hg@dom66.de> parents: 265 diff changeset	352 "%r" % filename)
8 7090c295c940 Two differend tree merge function implementations: not yet finished Franz Glasner <hg@dom66.de> parents: 5 diff changeset	353
7090c295c940 Two differend tree merge function implementations: not yet finished Franz Glasner <hg@dom66.de> parents: 5 diff changeset	354
11 aecb36d4025f Deactivate the "dict_merge()" function from yconfig Franz Glasner <f.glasner@feldmann-mg.com> parents: 10 diff changeset	355 if 0:
aecb36d4025f Deactivate the "dict_merge()" function from yconfig Franz Glasner <f.glasner@feldmann-mg.com> parents: 10 diff changeset	356 #
aecb36d4025f Deactivate the "dict_merge()" function from yconfig Franz Glasner <f.glasner@feldmann-mg.com> parents: 10 diff changeset	357 # From: https://github.com/jet9/python-yconfig/blob/master/yconfig.py
aecb36d4025f Deactivate the "dict_merge()" function from yconfig Franz Glasner <f.glasner@feldmann-mg.com> parents: 10 diff changeset	358 # License: BSD License
aecb36d4025f Deactivate the "dict_merge()" function from yconfig Franz Glasner <f.glasner@feldmann-mg.com> parents: 10 diff changeset	359 #
aecb36d4025f Deactivate the "dict_merge()" function from yconfig Franz Glasner <f.glasner@feldmann-mg.com> parents: 10 diff changeset	360 def dict_merge(a, b):
aecb36d4025f Deactivate the "dict_merge()" function from yconfig Franz Glasner <f.glasner@feldmann-mg.com> parents: 10 diff changeset	361 """Recursively merges dict's. not just simple a['key'] = b['key'], if
aecb36d4025f Deactivate the "dict_merge()" function from yconfig Franz Glasner <f.glasner@feldmann-mg.com> parents: 10 diff changeset	362 both a and bhave a key who's value is a dict then dict_merge is called
aecb36d4025f Deactivate the "dict_merge()" function from yconfig Franz Glasner <f.glasner@feldmann-mg.com> parents: 10 diff changeset	363 on both values and the result stored in the returned dictionary."""
8 7090c295c940 Two differend tree merge function implementations: not yet finished Franz Glasner <hg@dom66.de> parents: 5 diff changeset	364
11 aecb36d4025f Deactivate the "dict_merge()" function from yconfig Franz Glasner <f.glasner@feldmann-mg.com> parents: 10 diff changeset	365 if not isinstance(b, dict):
aecb36d4025f Deactivate the "dict_merge()" function from yconfig Franz Glasner <f.glasner@feldmann-mg.com> parents: 10 diff changeset	366 return b
207 b3b5ed34d180 Handle most flake8 errors and warnings. Franz Glasner <fzglas.hg@dom66.de> parents: 206 diff changeset	367 result = deepcopy(a) # noqa
11 aecb36d4025f Deactivate the "dict_merge()" function from yconfig Franz Glasner <f.glasner@feldmann-mg.com> parents: 10 diff changeset	368 for k, v in b.iteritems():
aecb36d4025f Deactivate the "dict_merge()" function from yconfig Franz Glasner <f.glasner@feldmann-mg.com> parents: 10 diff changeset	369 if k in result and isinstance(result[k], dict):
aecb36d4025f Deactivate the "dict_merge()" function from yconfig Franz Glasner <f.glasner@feldmann-mg.com> parents: 10 diff changeset	370 result[k] = dict_merge(result[k], v)
aecb36d4025f Deactivate the "dict_merge()" function from yconfig Franz Glasner <f.glasner@feldmann-mg.com> parents: 10 diff changeset	371 else:
207 b3b5ed34d180 Handle most flake8 errors and warnings. Franz Glasner <fzglas.hg@dom66.de> parents: 206 diff changeset	372 result[k] = deepcopy(v) # noqa
11 aecb36d4025f Deactivate the "dict_merge()" function from yconfig Franz Glasner <f.glasner@feldmann-mg.com> parents: 10 diff changeset	373 return result
8 7090c295c940 Two differend tree merge function implementations: not yet finished Franz Glasner <hg@dom66.de> parents: 5 diff changeset	374
7090c295c940 Two differend tree merge function implementations: not yet finished Franz Glasner <hg@dom66.de> parents: 5 diff changeset	375
144 7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	376 def merge(user, default, filter_comments=True):
110 29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions Franz Glasner <hg@dom66.de> parents: 108 diff changeset	377 """Logically merge the configuration in `user` into `default`.
98 d6ba53ce2091 Better documentation of the core function in "configmix" Franz Glasner <f.glasner@feldmann-mg.com> parents: 96 diff changeset	378
d6ba53ce2091 Better documentation of the core function in "configmix" Franz Glasner <f.glasner@feldmann-mg.com> parents: 96 diff changeset	379 :param ~configmix.config.Configuration user:
110 29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions Franz Glasner <hg@dom66.de> parents: 108 diff changeset	380 the new configuration that will be logically merged
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions Franz Glasner <hg@dom66.de> parents: 108 diff changeset	381 into `default`
98 d6ba53ce2091 Better documentation of the core function in "configmix" Franz Glasner <f.glasner@feldmann-mg.com> parents: 96 diff changeset	382 :param ~configmix.config.Configuration default:
110 29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions Franz Glasner <hg@dom66.de> parents: 108 diff changeset	383 the base configuration where `user` is logically merged into
144 7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	384 :param bool filter_comments: flag whether to filter comment keys that
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	385 start with any of the items in :data:`COMMENTS`
110 29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions Franz Glasner <hg@dom66.de> parents: 108 diff changeset	386 :returns: `user` with the necessary amendments from `default`.
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions Franz Glasner <hg@dom66.de> parents: 108 diff changeset	387 If `user` is ``None`` then `default` is returned.
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions Franz Glasner <hg@dom66.de> parents: 108 diff changeset	388
144 7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	389 .. note:: The configuration in `user` is augmented/changed
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	390 inplace.
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	391
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	392 The configuration in `default` will be changed inplace
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	393 when filtering out comments (which is the default).
8 7090c295c940 Two differend tree merge function implementations: not yet finished Franz Glasner <hg@dom66.de> parents: 5 diff changeset	394
7090c295c940 Two differend tree merge function implementations: not yet finished Franz Glasner <hg@dom66.de> parents: 5 diff changeset	395 From http://stackoverflow.com/questions/823196/yaml-merge-in-python
7090c295c940 Two differend tree merge function implementations: not yet finished Franz Glasner <hg@dom66.de> parents: 5 diff changeset	396
7090c295c940 Two differend tree merge function implementations: not yet finished Franz Glasner <hg@dom66.de> parents: 5 diff changeset	397 """
110 29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions Franz Glasner <hg@dom66.de> parents: 108 diff changeset	398 if user is None:
144 7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	399 if filter_comments:
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	400 _filter_comments(default)
13 24ba462b9b4b Return the `default' argument when the given `user' argument is `None' and when it is the first call on merge Franz Glasner <f.glasner@feldmann-mg.com> parents: 11 diff changeset	401 return default
144 7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	402 if filter_comments:
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	403 _filter_comments(user)
8 7090c295c940 Two differend tree merge function implementations: not yet finished Franz Glasner <hg@dom66.de> parents: 5 diff changeset	404 if isinstance(user, dict) and isinstance(default, dict):
9 6835a5663008 FIX: Style Franz Glasner <f.glasner@feldmann-mg.com> parents: 8 diff changeset	405 for k, v in default.items():
144 7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	406 if filter_comments and _is_comment(k):
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	407 continue
8 7090c295c940 Two differend tree merge function implementations: not yet finished Franz Glasner <hg@dom66.de> parents: 5 diff changeset	408 if k not in user:
7090c295c940 Two differend tree merge function implementations: not yet finished Franz Glasner <hg@dom66.de> parents: 5 diff changeset	409 user[k] = v
7090c295c940 Two differend tree merge function implementations: not yet finished Franz Glasner <hg@dom66.de> parents: 5 diff changeset	410 else:
144 7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	411 user[k] = _merge(user[k], v, filter_comments)
8 7090c295c940 Two differend tree merge function implementations: not yet finished Franz Glasner <hg@dom66.de> parents: 5 diff changeset	412 return user
10 58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers Franz Glasner <f.glasner@feldmann-mg.com> parents: 9 diff changeset	413
58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers Franz Glasner <f.glasner@feldmann-mg.com> parents: 9 diff changeset	414
144 7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	415 def _merge(user, default, filter_comments):
110 29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions Franz Glasner <hg@dom66.de> parents: 108 diff changeset	416 """Recursion helper for :meth:`merge`
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions Franz Glasner <hg@dom66.de> parents: 108 diff changeset	417
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions Franz Glasner <hg@dom66.de> parents: 108 diff changeset	418 """
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions Franz Glasner <hg@dom66.de> parents: 108 diff changeset	419 if isinstance(user, dict) and isinstance(default, dict):
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions Franz Glasner <hg@dom66.de> parents: 108 diff changeset	420 for k, v in default.items():
144 7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	421 if filter_comments and _is_comment(k):
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	422 continue
110 29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions Franz Glasner <hg@dom66.de> parents: 108 diff changeset	423 if k not in user:
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions Franz Glasner <hg@dom66.de> parents: 108 diff changeset	424 user[k] = v
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions Franz Glasner <hg@dom66.de> parents: 108 diff changeset	425 else:
144 7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	426 user[k] = _merge(user[k], v, filter_comments)
110 29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions Franz Glasner <hg@dom66.de> parents: 108 diff changeset	427 return user
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions Franz Glasner <hg@dom66.de> parents: 108 diff changeset	428
29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions Franz Glasner <hg@dom66.de> parents: 108 diff changeset	429
144 7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	430 def safe_merge(user, default, filter_comments=True):
111 d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging. Franz Glasner <hg@dom66.de> parents: 110 diff changeset	431 """A more safe version of :func:`merge` that makes deep copies of
13 24ba462b9b4b Return the `default' argument when the given `user' argument is `None' and when it is the first call on merge Franz Glasner <f.glasner@feldmann-mg.com> parents: 11 diff changeset	432 the returned container objects.
24ba462b9b4b Return the `default' argument when the given `user' argument is `None' and when it is the first call on merge Franz Glasner <f.glasner@feldmann-mg.com> parents: 11 diff changeset	433
144 7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	434 Contrary to :func:`merge` no given argument is ever changed
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	435 inplace. Every object from `default` is decoupled from the result
165 6ca90e80f4f4 FIX: Docu: wording Franz Glasner <fzglas.hg@dom66.de> parents: 163 diff changeset	436 -- so changing the `default` configuration later does not propagate
144 7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	437 into a merged configuration later.
111 d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging. Franz Glasner <hg@dom66.de> parents: 110 diff changeset	438
10 58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers Franz Glasner <f.glasner@feldmann-mg.com> parents: 9 diff changeset	439 """
110 29cf359ddf4d Remove the "_first" parameter from "merge" and "safe_merge" by splitting into two functions Franz Glasner <hg@dom66.de> parents: 108 diff changeset	440 if user is None:
144 7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	441 if filter_comments:
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	442 _filter_comments(default)
111 d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging. Franz Glasner <hg@dom66.de> parents: 110 diff changeset	443 return copy.deepcopy(default)
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging. Franz Glasner <hg@dom66.de> parents: 110 diff changeset	444 user = copy.deepcopy(user)
144 7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	445 if filter_comments:
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	446 _filter_comments(user)
10 58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers Franz Glasner <f.glasner@feldmann-mg.com> parents: 9 diff changeset	447 if isinstance(user, dict) and isinstance(default, dict):
58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers Franz Glasner <f.glasner@feldmann-mg.com> parents: 9 diff changeset	448 for k, v in default.items():
144 7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	449 if filter_comments and _is_comment(k):
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	450 continue
10 58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers Franz Glasner <f.glasner@feldmann-mg.com> parents: 9 diff changeset	451 if k not in user:
111 d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging. Franz Glasner <hg@dom66.de> parents: 110 diff changeset	452 user[k] = copy.deepcopy(v)
10 58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers Franz Glasner <f.glasner@feldmann-mg.com> parents: 9 diff changeset	453 else:
144 7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	454 user[k] = _safe_merge(user[k], v, filter_comments)
10 58af59d5af40 A "safe_merge" that makes (shallow) copies instead of directly manipulating given containers Franz Glasner <f.glasner@feldmann-mg.com> parents: 9 diff changeset	455 return user
111 d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging. Franz Glasner <hg@dom66.de> parents: 110 diff changeset	456
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging. Franz Glasner <hg@dom66.de> parents: 110 diff changeset	457
144 7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	458 def _safe_merge(user, default, filter_comments):
111 d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging. Franz Glasner <hg@dom66.de> parents: 110 diff changeset	459 """Recursion helper for :meth:`safe_merge`
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging. Franz Glasner <hg@dom66.de> parents: 110 diff changeset	460
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging. Franz Glasner <hg@dom66.de> parents: 110 diff changeset	461 """
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging. Franz Glasner <hg@dom66.de> parents: 110 diff changeset	462 if isinstance(user, dict) and isinstance(default, dict):
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging. Franz Glasner <hg@dom66.de> parents: 110 diff changeset	463 for k, v in default.items():
144 7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	464 if filter_comments and _is_comment(k):
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	465 continue
111 d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging. Franz Glasner <hg@dom66.de> parents: 110 diff changeset	466 if k not in user:
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging. Franz Glasner <hg@dom66.de> parents: 110 diff changeset	467 user[k] = copy.deepcopy(v)
d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging. Franz Glasner <hg@dom66.de> parents: 110 diff changeset	468 else:
144 7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	469 user[k] = _safe_merge(user[k], v, filter_comments)
111 d51a18e5b0e3 Reimplement configmix.safe_merge() do to a deepcopy of all source configurations when merging. Franz Glasner <hg@dom66.de> parents: 110 diff changeset	470 return user
139 c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles Franz Glasner <hg@dom66.de> parents: 138 diff changeset	471
c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles Franz Glasner <hg@dom66.de> parents: 138 diff changeset	472
144 7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	473 def _filter_comments(d):
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	474 """Recursively filter comments keys in the dict `d`.
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	475
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	476 Comment keys are keys that start with any of the items in
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	477 :data:`COMMENTS`.
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	478
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	479 """
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	480 if not isinstance(d, dict):
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	481 return
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	482 # use a copy of the keys because we change `d` while iterating
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	483 for k in list(d.keys()):
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	484 if _is_comment(k):
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	485 del d[k]
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	486 else:
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	487 if isinstance(d[k], dict):
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	488 _filter_comments(d[k])
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	489
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	490
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	491 def _is_comment(k):
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	492 for i in COMMENTS:
256 7e26d31f52de FIX: Allow non-text keys when merging: handle .startswith() errors gracefully Franz Glasner <f.glasner@feldmann-mg.com> parents: 253 diff changeset	493 try:
7e26d31f52de FIX: Allow non-text keys when merging: handle .startswith() errors gracefully Franz Glasner <f.glasner@feldmann-mg.com> parents: 253 diff changeset	494 if k.startswith(i):
7e26d31f52de FIX: Allow non-text keys when merging: handle .startswith() errors gracefully Franz Glasner <f.glasner@feldmann-mg.com> parents: 253 diff changeset	495 return True
7e26d31f52de FIX: Allow non-text keys when merging: handle .startswith() errors gracefully Franz Glasner <f.glasner@feldmann-mg.com> parents: 253 diff changeset	496 except AttributeError:
7e26d31f52de FIX: Allow non-text keys when merging: handle .startswith() errors gracefully Franz Glasner <f.glasner@feldmann-mg.com> parents: 253 diff changeset	497 # non-string key
7e26d31f52de FIX: Allow non-text keys when merging: handle .startswith() errors gracefully Franz Glasner <f.glasner@feldmann-mg.com> parents: 253 diff changeset	498 return False
144 7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	499 return False
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	500
7e6ec99d5ff5 Allow comments as keys and filter them by default Franz Glasner <hg@dom66.de> parents: 141 diff changeset	501
139 c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles Franz Glasner <hg@dom66.de> parents: 138 diff changeset	502 #
178 eeb3ed104ea1 Begin refactoring the associations between filename extensions and loader functions: Franz Glasner <fzglas.hg@dom66.de> parents: 177 diff changeset	503 # Init loader defaults: mode->loader and extension->mode
139 c87b0dc54e1d Allow custom configuration filename extensions and custom loaders that can handle custom configuration file syntax styles Franz Glasner <hg@dom66.de> parents: 138 diff changeset	504 #
189 0d0980ed74cc Rename the "_mode_loaders" variable to "mode_loaders" and make is therefore a "public" item Franz Glasner <fzglas.hg@dom66.de> parents: 188 diff changeset	505 mode_loaders.update(DEFAULT_MODE_LOADERS)
181 7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search Franz Glasner <fzglas.hg@dom66.de> parents: 180 diff changeset	506 for _pattern, _mode in DEFAULT_ASSOC:
7cfdc972af42 Refactor: Renamed public functions to be conform with the new loader search Franz Glasner <fzglas.hg@dom66.de> parents: 180 diff changeset	507 set_assoc(_pattern, _mode)

Mercurial > hgrepos > Python > libs > ConfigMix

annotate configmix/__init__.py @ 267:d715c10f2930

annotate configmix/init.py @ 267:d715c10f2930