Mercurial > hgrepos > Python > libs > ConfigMix

annotate configmix/yaml.py @ 654:0d6673d06c2c

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Add support for using "tomllib" (in Python's stdlib since 3.11) and "tomli" TOML packages. They are preferred if they are found to be installed. But note that the declared dependency for the "toml" extra nevertheless is the "toml" package. Because it is available for all supported Python versions. So use Python 3.11+ or install "tomli" manually if you want to use the alternate packages.
author Franz Glasner <fzglas.hg@dom66.de>
date Thu, 19 May 2022 22:10:59 +0200
parents f454889e41fa
children 80d203ed3556
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
1 # -*- coding: utf-8 -*-
208
bbe8513ea649 Handle flake8 E265 "block comment should start with '# ': use '# :-' instead of '#-' to mark copyright and license comments
Franz Glasner <fzglas.hg@dom66.de>
parents: 207
diff changeset
2 # :-
593
f454889e41fa Adjust copyright year (the end) to 2022
Franz Glasner <fzglas.hg@dom66.de>
parents: 296
diff changeset
3 # :Copyright: (c) 2015-2022, Franz Glasner. All rights reserved.
296
eed16a1ec8f3 Use SPDX license identifiers (either full or short) all over the package
Franz Glasner <fzglas.hg@dom66.de>
parents: 291
diff changeset
4 # :License: BSD-3-Clause. See LICENSE.txt for details.
208
bbe8513ea649 Handle flake8 E265 "block comment should start with '# ': use '# :-' instead of '#-' to mark copyright and license comments
Franz Glasner <fzglas.hg@dom66.de>
parents: 207
diff changeset
5 # :-
90
99e7b10c8aa8 Mark the yaml module with ":mod:"
Franz Glasner <hg@dom66.de>
parents: 82
diff changeset
6 """Simple wrapper for :mod:`yaml` to support all-unicode strings when
99e7b10c8aa8 Mark the yaml module with ":mod:"
Franz Glasner <hg@dom66.de>
parents: 82
diff changeset
7 loading configuration files.
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
8
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
9 """
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
10
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
11 from __future__ import division, print_function, absolute_import
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
12
250
ff964825a75a Style: placement of "__all__"
Franz Glasner <fzglas.hg@dom66.de>
parents: 237
diff changeset
13
ff964825a75a Style: placement of "__all__"
Franz Glasner <fzglas.hg@dom66.de>
parents: 237
diff changeset
14 __all__ = ["safe_load", "safe_load_all", "load", "load_all"]
ff964825a75a Style: placement of "__all__"
Franz Glasner <fzglas.hg@dom66.de>
parents: 237
diff changeset
15
ff964825a75a Style: placement of "__all__"
Franz Glasner <fzglas.hg@dom66.de>
parents: 237
diff changeset
16
3
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
17 try:
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
18 from collections import OrderedDict
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
19 except ImportError:
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
20 try:
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
21 from ordereddict import OrderedDict
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
22 except ImportError:
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
23 OrderedDict = None
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
24 import yaml
3
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
25 import yaml.constructor
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
26
21
ce290b10dac5 Better Py2/Py3 compatibility: mark some strings explicitly as Unicode
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 5
diff changeset
27 from .compat import u
ce290b10dac5 Better Py2/Py3 compatibility: mark some strings explicitly as Unicode
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 5
diff changeset
28
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
29
136
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
30 DictImpl = OrderedDict or dict
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
31
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
32
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
33 class ConfigLoader(yaml.Loader):
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
34
56
1f11672c4615 Optimize the documentation: make references working with Sphinx using :role:`target`
Franz Glasner <hg@dom66.de>
parents: 54
diff changeset
35 """A YAML loader, which makes all ``!!str`` strings to Unicode.
1f11672c4615 Optimize the documentation: make references working with Sphinx using :role:`target`
Franz Glasner <hg@dom66.de>
parents: 54
diff changeset
36 Standard PyYAML does this only in the non-ASCII case.
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
37
3
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
38 If an `OrderedDict` implementation is available then all "map" and
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
39 "omap" nodes are constructed as `OrderedDict`.
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
40 This is against YAML specs but within configuration files it seems
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
41 more natural.
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
42
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
43 """
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
44
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
45 def __init__(self, *args, **kwds):
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
46 strict = kwds.pop("strict", False)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
47 self.__allow_duplicate_keys = not strict
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
48 yaml.Loader.__init__(self, *args, **kwds)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
49
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
50 def construct_yaml_str(self, node):
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
51 return self.construct_scalar(node)
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
52
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
53 #
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
54 # From https://pypi.python.org/pypi/yamlordereddictloader/0.1.1
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
55 # (MIT License)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
56 #
3
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
57
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
58 def construct_yaml_map(self, node):
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
59 data = DictImpl()
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
60 yield data
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
61 value = self.construct_mapping(node)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
62 data.update(value)
3
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
63
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
64 def construct_mapping(self, node, deep=False):
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
65 if isinstance(node, yaml.MappingNode):
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
66 self.flatten_mapping(node)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
67 else:
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
68 raise yaml.constructor.ConstructorError(
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
69 None,
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
70 None,
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
71 'expected a mapping node, but found %s' % node.id,
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
72 node.start_mark)
3
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
73
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
74 mapping = DictImpl()
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
75 for key_node, value_node in node.value:
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
76 key = self.construct_object(key_node, deep=deep)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
77 try:
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
78 hash(key)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
79 except TypeError as err:
207
b3b5ed34d180 Handle most flake8 errors and warnings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
80 raise yaml.constructor.ConstructorError(
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
81 'while constructing a mapping', node.start_mark,
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
82 'found unacceptable key (%s)' % (err,
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
83 key_node.start_mark)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
84 )
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
85 value = self.construct_object(value_node, deep=deep)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
86 if not self.__allow_duplicate_keys and key in mapping:
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
87 raise yaml.constructor.ConstructorError(
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
88 'while constructing a mapping', node.start_mark,
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
89 'found duplicate key %r (%s)' % (key,
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
90 key_node.start_mark)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
91 )
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
92 mapping[key] = value
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
93 return mapping
3
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
94
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
95
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
96 ConfigLoader.add_constructor(
21
ce290b10dac5 Better Py2/Py3 compatibility: mark some strings explicitly as Unicode
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 5
diff changeset
97 u("tag:yaml.org,2002:str"),
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
98 ConfigLoader.construct_yaml_str)
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
99 ConfigLoader.add_constructor(
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
100 u("tag:yaml.org,2002:map"),
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
101 ConfigLoader.construct_yaml_map)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
102 ConfigLoader.add_constructor(
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
103 u("tag:yaml.org,2002:omap"),
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
104 ConfigLoader.construct_yaml_map)
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
105
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
106
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
107 class ConfigSafeLoader(yaml.SafeLoader):
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
108
56
1f11672c4615 Optimize the documentation: make references working with Sphinx using :role:`target`
Franz Glasner <hg@dom66.de>
parents: 54
diff changeset
109 """A safe YAML loader, which makes all ``!!str`` strings to Unicode.
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
110 Standard PyYAML does this only in the non-ASCII case.
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
111
3
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
112 If an `OrderedDict` implementation is available then all "map" and
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
113 "omap" nodes are constructed as `OrderedDict`.
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
114 This is against YAML specs but within configuration files it seems
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
115 more natural.
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
116
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
117 """
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
118
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
119 def __init__(self, *args, **kwds):
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
120 strict = kwds.pop("strict", False)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
121 self.__allow_duplicate_keys = not strict
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
122 yaml.SafeLoader.__init__(self, *args, **kwds)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
123
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
124 def construct_yaml_str(self, node):
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
125 return self.construct_scalar(node)
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
126
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
127 #
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
128 # From https://pypi.python.org/pypi/yamlordereddictloader/0.1.1
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
129 # (MIT License)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
130 #
3
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
131
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
132 def construct_yaml_map(self, node):
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
133 data = DictImpl()
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
134 yield data
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
135 value = self.construct_mapping(node)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
136 data.update(value)
3
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
137
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
138 def construct_mapping(self, node, deep=False):
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
139 if isinstance(node, yaml.MappingNode):
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
140 self.flatten_mapping(node)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
141 else:
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
142 raise yaml.constructor.ConstructorError(
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
143 None,
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
144 None,
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
145 'expected a mapping node, but found %s' % node.id,
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
146 node.start_mark)
3
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
147
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
148 mapping = DictImpl()
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
149 for key_node, value_node in node.value:
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
150 key = self.construct_object(key_node, deep=deep)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
151 try:
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
152 hash(key)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
153 except TypeError as err:
207
b3b5ed34d180 Handle most flake8 errors and warnings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
154 raise yaml.constructor.ConstructorError(
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
155 'while constructing a mapping', node.start_mark,
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
156 'found unacceptable key (%s)' % (err,
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
157 key_node.start_mark)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
158 )
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
159 value = self.construct_object(value_node, deep=deep)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
160 if not self.__allow_duplicate_keys and key in mapping:
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
161 raise yaml.constructor.ConstructorError(
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
162 'while constructing a mapping', node.start_mark,
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
163 'found duplicate key %r (%s)' % (key,
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
164 key_node.start_mark)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
165 )
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
166 mapping[key] = value
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
167 return mapping
3
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
168
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
169
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
170 ConfigSafeLoader.add_constructor(
21
ce290b10dac5 Better Py2/Py3 compatibility: mark some strings explicitly as Unicode
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 5
diff changeset
171 u("tag:yaml.org,2002:str"),
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
172 ConfigSafeLoader.construct_yaml_str)
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
173 ConfigSafeLoader.add_constructor(
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
174 u("tag:yaml.org,2002:map"),
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
175 ConfigSafeLoader.construct_yaml_map)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
176 ConfigSafeLoader.add_constructor(
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
177 u("tag:yaml.org,2002:omap"),
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
178 ConfigSafeLoader.construct_yaml_map)
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
179
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
180
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
181 def config_loader_factory(strict=False):
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
182 def _real_factory(*args, **kwds):
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
183 kwds["strict"] = strict
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
184 return ConfigLoader(*args, **kwds)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
185 return _real_factory
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
186
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
187
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
188 def config_safe_loader_factory(strict=False):
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
189 def _real_factory(*args, **kwds):
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
190 kwds["strict"] = strict
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
191 return ConfigSafeLoader(*args, **kwds)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
192 return _real_factory
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
193
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
194
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
195 def load(stream, Loader=None, strict=False):
135
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
196 """Parse the given `stream` and return a Python object constructed
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
197 from for the first document in the stream.
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
198
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
199 If `strict` is `True` then duplicate mapping keys within a YAML
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
200 document are detected and prevented. If a `Loader` is given then
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
201 `strict` does not apply.
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
202
135
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
203 """
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
204 if Loader is None:
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
205 Loader = config_loader_factory(strict=strict)
134
2f2e819e8d17 Check the return type of the JSON and YAML loading functions: they must be a dict alike
Franz Glasner <hg@dom66.de>
parents: 90
diff changeset
206 data = yaml.load(stream, Loader)
136
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
207 # Map an empty document to an empty dict
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
208 if data is None:
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
209 return DictImpl()
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
210 if not isinstance(data, DictImpl):
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
211 raise TypeError("YAML root object must be a mapping")
134
2f2e819e8d17 Check the return type of the JSON and YAML loading functions: they must be a dict alike
Franz Glasner <hg@dom66.de>
parents: 90
diff changeset
212 return data
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
213
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
214
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
215 def load_all(stream, Loader=None, strict=False):
135
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
216 """Parse the given `stream` and return a sequence of Python objects
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
217 corresponding to the documents in the `stream`.
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
218
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
219 If `strict` is `True` then duplicate mapping keys within a YAML
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
220 document are detected and prevented. If a `Loader` is given then
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
221 `strict` does not apply.
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
222
135
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
223 """
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
224 if Loader is None:
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
225 Loader = config_loader_factory(strict=strict)
134
2f2e819e8d17 Check the return type of the JSON and YAML loading functions: they must be a dict alike
Franz Glasner <hg@dom66.de>
parents: 90
diff changeset
226 data_all = yaml.load_all(stream, Loader)
136
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
227 rdata = []
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
228 for data in data_all:
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
229 if data is None:
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
230 rdata.append(DictImpl())
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
231 else:
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
232 if not isinstance(data, DictImpl):
134
2f2e819e8d17 Check the return type of the JSON and YAML loading functions: they must be a dict alike
Franz Glasner <hg@dom66.de>
parents: 90
diff changeset
233 raise TypeError("YAML root object must be a mapping")
136
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
234 rdata.append(data)
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
235 return rdata
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
236
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
237
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
238 def safe_load(stream, strict=False):
135
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
239 """Parse the given `stream` and return a Python object constructed
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
240 from for the first document in the stream.
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
241
136
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
242 Recognizes only standard YAML tags and cannot construct an
135
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
243 arbitrary Python object.
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
244
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
245 If `strict` is `True` then duplicate mapping keys within a YAML document
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
246 are detected and prevented.
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
247
135
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
248 """
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
249 data = yaml.load(stream,
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
250 Loader=config_safe_loader_factory(strict=strict))
136
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
251 # Map an empty document to an empty dict
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
252 if data is None:
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
253 return DictImpl()
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
254 if not isinstance(data, DictImpl):
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
255 raise TypeError("YAML root object must be a mapping")
134
2f2e819e8d17 Check the return type of the JSON and YAML loading functions: they must be a dict alike
Franz Glasner <hg@dom66.de>
parents: 90
diff changeset
256 return data
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
257
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
258
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
259 def safe_load_all(stream, strict=False):
135
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
260 """Return the list of all decoded YAML documents in the file `stream`.
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
261
136
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
262 Recognizes only standard YAML tags and cannot construct an
135
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
263 arbitrary Python object.
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
264
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
265 If `strict` is `True` then duplicate mapping keys within a YAML document
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
266 are detected and prevented.
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
267
135
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
268 """
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
269 data_all = yaml.load_all(stream,
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
270 Loader=config_safe_loader_factory(strict=strict))
136
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
271 rdata = []
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
272 for data in data_all:
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
273 if data is None:
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
274 rdata.append(DictImpl())
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
275 else:
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
276 if not isinstance(data, DictImpl):
134
2f2e819e8d17 Check the return type of the JSON and YAML loading functions: they must be a dict alike
Franz Glasner <hg@dom66.de>
parents: 90
diff changeset
277 raise TypeError("YAML root object must be a mapping")
136
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
278 rdata.append(data)
134
2f2e819e8d17 Check the return type of the JSON and YAML loading functions: they must be a dict alike
Franz Glasner <hg@dom66.de>
parents: 90
diff changeset
279 return data_all