Mercurial > hgrepos > Python > libs > ConfigMix

annotate configmix/yaml.py @ 760:1514e7d2f691

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
FIX: add forgotten test configuration data file
author Franz Glasner <fzglas.hg@dom66.de>
date Thu, 07 Dec 2023 08:40:46 +0100
parents 80d203ed3556
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
1 # -*- coding: utf-8 -*-
208
bbe8513ea649 Handle flake8 E265 "block comment should start with '# ': use '# :-' instead of '#-' to mark copyright and license comments
Franz Glasner <fzglas.hg@dom66.de>
parents: 207
diff changeset
2 # :-
593
f454889e41fa Adjust copyright year (the end) to 2022
Franz Glasner <fzglas.hg@dom66.de>
parents: 296
diff changeset
3 # :Copyright: (c) 2015-2022, Franz Glasner. All rights reserved.
296
eed16a1ec8f3 Use SPDX license identifiers (either full or short) all over the package
Franz Glasner <fzglas.hg@dom66.de>
parents: 291
diff changeset
4 # :License: BSD-3-Clause. See LICENSE.txt for details.
208
bbe8513ea649 Handle flake8 E265 "block comment should start with '# ': use '# :-' instead of '#-' to mark copyright and license comments
Franz Glasner <fzglas.hg@dom66.de>
parents: 207
diff changeset
5 # :-
90
99e7b10c8aa8 Mark the yaml module with ":mod:"
Franz Glasner <hg@dom66.de>
parents: 82
diff changeset
6 """Simple wrapper for :mod:`yaml` to support all-unicode strings when
99e7b10c8aa8 Mark the yaml module with ":mod:"
Franz Glasner <hg@dom66.de>
parents: 82
diff changeset
7 loading configuration files.
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
8
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
9 """
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
10
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
11 from __future__ import division, print_function, absolute_import
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
12
250
ff964825a75a Style: placement of "__all__"
Franz Glasner <fzglas.hg@dom66.de>
parents: 237
diff changeset
13
ff964825a75a Style: placement of "__all__"
Franz Glasner <fzglas.hg@dom66.de>
parents: 237
diff changeset
14 __all__ = ["safe_load", "safe_load_all", "load", "load_all"]
ff964825a75a Style: placement of "__all__"
Franz Glasner <fzglas.hg@dom66.de>
parents: 237
diff changeset
15
ff964825a75a Style: placement of "__all__"
Franz Glasner <fzglas.hg@dom66.de>
parents: 237
diff changeset
16
3
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
17 try:
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
18 from collections import OrderedDict
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
19 except ImportError:
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
20 try:
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
21 from ordereddict import OrderedDict
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
22 except ImportError:
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
23 OrderedDict = None
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
24 import yaml
3
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
25 import yaml.constructor
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
26
21
ce290b10dac5 Better Py2/Py3 compatibility: mark some strings explicitly as Unicode
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 5
diff changeset
27 from .compat import u
ce290b10dac5 Better Py2/Py3 compatibility: mark some strings explicitly as Unicode
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 5
diff changeset
28
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
29
136
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
30 DictImpl = OrderedDict or dict
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
31
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
32
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
33 class ConfigLoader(yaml.Loader):
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
34
56
1f11672c4615 Optimize the documentation: make references working with Sphinx using :role:`target`
Franz Glasner <hg@dom66.de>
parents: 54
diff changeset
35 """A YAML loader, which makes all ``!!str`` strings to Unicode.
1f11672c4615 Optimize the documentation: make references working with Sphinx using :role:`target`
Franz Glasner <hg@dom66.de>
parents: 54
diff changeset
36 Standard PyYAML does this only in the non-ASCII case.
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
37
3
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
38 If an `OrderedDict` implementation is available then all "map" and
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
39 "omap" nodes are constructed as `OrderedDict`.
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
40 This is against YAML specs but within configuration files it seems
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
41 more natural.
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
42
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
43 """
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
44
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
45 def __init__(self, *args, **kwds):
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
46 strict = kwds.pop("strict", False)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
47 self.__allow_duplicate_keys = not strict
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
48 yaml.Loader.__init__(self, *args, **kwds)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
49
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
50 def construct_yaml_str(self, node):
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
51 return self.construct_scalar(node)
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
52
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
53 #
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
54 # From https://pypi.python.org/pypi/yamlordereddictloader/0.1.1
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
55 # (MIT License)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
56 #
3
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
57
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
58 def construct_yaml_map(self, node):
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
59 data = DictImpl()
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
60 yield data
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
61 value = self.construct_mapping(node)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
62 data.update(value)
3
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
63
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
64 def construct_mapping(self, node, deep=False):
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
65 if isinstance(node, yaml.MappingNode):
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
66 self.flatten_mapping(node)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
67 else:
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
68 raise yaml.constructor.ConstructorError(
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
69 None,
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
70 None,
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
71 'expected a mapping node, but found %s' % node.id,
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
72 node.start_mark)
3
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
73
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
74 mapping = DictImpl()
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
75 for key_node, value_node in node.value:
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
76 key = self.construct_object(key_node, deep=deep)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
77 try:
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
78 hash(key)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
79 except TypeError as err:
207
b3b5ed34d180 Handle most flake8 errors and warnings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
80 raise yaml.constructor.ConstructorError(
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
81 'while constructing a mapping', node.start_mark,
686
80d203ed3556 FIX: The YAML loader had wrong format strings in its exception handlers.
Franz Glasner <fzglas.hg@dom66.de>
parents: 593
diff changeset
82 'found unacceptable key (%s)' % (err, ),
80d203ed3556 FIX: The YAML loader had wrong format strings in its exception handlers.
Franz Glasner <fzglas.hg@dom66.de>
parents: 593
diff changeset
83 key_node.start_mark)
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
84 value = self.construct_object(value_node, deep=deep)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
85 if not self.__allow_duplicate_keys and key in mapping:
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
86 raise yaml.constructor.ConstructorError(
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
87 'while constructing a mapping', node.start_mark,
686
80d203ed3556 FIX: The YAML loader had wrong format strings in its exception handlers.
Franz Glasner <fzglas.hg@dom66.de>
parents: 593
diff changeset
88 'found duplicate key %r' % (key, ),
80d203ed3556 FIX: The YAML loader had wrong format strings in its exception handlers.
Franz Glasner <fzglas.hg@dom66.de>
parents: 593
diff changeset
89 key_node.start_mark)
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
90 mapping[key] = value
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
91 return mapping
3
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
92
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
93
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
94 ConfigLoader.add_constructor(
21
ce290b10dac5 Better Py2/Py3 compatibility: mark some strings explicitly as Unicode
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 5
diff changeset
95 u("tag:yaml.org,2002:str"),
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
96 ConfigLoader.construct_yaml_str)
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
97 ConfigLoader.add_constructor(
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
98 u("tag:yaml.org,2002:map"),
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
99 ConfigLoader.construct_yaml_map)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
100 ConfigLoader.add_constructor(
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
101 u("tag:yaml.org,2002:omap"),
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
102 ConfigLoader.construct_yaml_map)
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
103
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
104
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
105 class ConfigSafeLoader(yaml.SafeLoader):
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
106
56
1f11672c4615 Optimize the documentation: make references working with Sphinx using :role:`target`
Franz Glasner <hg@dom66.de>
parents: 54
diff changeset
107 """A safe YAML loader, which makes all ``!!str`` strings to Unicode.
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
108 Standard PyYAML does this only in the non-ASCII case.
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
109
3
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
110 If an `OrderedDict` implementation is available then all "map" and
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
111 "omap" nodes are constructed as `OrderedDict`.
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
112 This is against YAML specs but within configuration files it seems
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
113 more natural.
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
114
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
115 """
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
116
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
117 def __init__(self, *args, **kwds):
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
118 strict = kwds.pop("strict", False)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
119 self.__allow_duplicate_keys = not strict
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
120 yaml.SafeLoader.__init__(self, *args, **kwds)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
121
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
122 def construct_yaml_str(self, node):
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
123 return self.construct_scalar(node)
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
124
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
125 #
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
126 # From https://pypi.python.org/pypi/yamlordereddictloader/0.1.1
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
127 # (MIT License)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
128 #
3
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
129
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
130 def construct_yaml_map(self, node):
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
131 data = DictImpl()
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
132 yield data
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
133 value = self.construct_mapping(node)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
134 data.update(value)
3
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
135
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
136 def construct_mapping(self, node, deep=False):
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
137 if isinstance(node, yaml.MappingNode):
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
138 self.flatten_mapping(node)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
139 else:
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
140 raise yaml.constructor.ConstructorError(
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
141 None,
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
142 None,
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
143 'expected a mapping node, but found %s' % node.id,
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
144 node.start_mark)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
145 mapping = DictImpl()
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
146 for key_node, value_node in node.value:
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
147 key = self.construct_object(key_node, deep=deep)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
148 try:
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
149 hash(key)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
150 except TypeError as err:
207
b3b5ed34d180 Handle most flake8 errors and warnings.
Franz Glasner <fzglas.hg@dom66.de>
parents: 156
diff changeset
151 raise yaml.constructor.ConstructorError(
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
152 'while constructing a mapping', node.start_mark,
686
80d203ed3556 FIX: The YAML loader had wrong format strings in its exception handlers.
Franz Glasner <fzglas.hg@dom66.de>
parents: 593
diff changeset
153 'found unacceptable key (%s)' % (err, ),
80d203ed3556 FIX: The YAML loader had wrong format strings in its exception handlers.
Franz Glasner <fzglas.hg@dom66.de>
parents: 593
diff changeset
154 key_node.start_mark)
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
155 value = self.construct_object(value_node, deep=deep)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
156 if not self.__allow_duplicate_keys and key in mapping:
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
157 raise yaml.constructor.ConstructorError(
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
158 'while constructing a mapping', node.start_mark,
686
80d203ed3556 FIX: The YAML loader had wrong format strings in its exception handlers.
Franz Glasner <fzglas.hg@dom66.de>
parents: 593
diff changeset
159 'found duplicate key %r' % (key, ),
80d203ed3556 FIX: The YAML loader had wrong format strings in its exception handlers.
Franz Glasner <fzglas.hg@dom66.de>
parents: 593
diff changeset
160 key_node.start_mark)
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
161 mapping[key] = value
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
162 return mapping
3
bedc4f95b9e9 Use a YAML constructor that automatically creates OrderedDict objects when an OrderedDict implementation is available
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 1
diff changeset
163
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
164
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
165 ConfigSafeLoader.add_constructor(
21
ce290b10dac5 Better Py2/Py3 compatibility: mark some strings explicitly as Unicode
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 5
diff changeset
166 u("tag:yaml.org,2002:str"),
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
167 ConfigSafeLoader.construct_yaml_str)
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
168 ConfigSafeLoader.add_constructor(
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
169 u("tag:yaml.org,2002:map"),
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
170 ConfigSafeLoader.construct_yaml_map)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
171 ConfigSafeLoader.add_constructor(
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
172 u("tag:yaml.org,2002:omap"),
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
173 ConfigSafeLoader.construct_yaml_map)
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
174
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
175
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
176 def config_loader_factory(strict=False):
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
177 def _real_factory(*args, **kwds):
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
178 kwds["strict"] = strict
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
179 return ConfigLoader(*args, **kwds)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
180 return _real_factory
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
181
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
182
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
183 def config_safe_loader_factory(strict=False):
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
184 def _real_factory(*args, **kwds):
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
185 kwds["strict"] = strict
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
186 return ConfigSafeLoader(*args, **kwds)
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
187 return _real_factory
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
188
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
189
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
190 def load(stream, Loader=None, strict=False):
135
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
191 """Parse the given `stream` and return a Python object constructed
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
192 from for the first document in the stream.
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
193
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
194 If `strict` is `True` then duplicate mapping keys within a YAML
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
195 document are detected and prevented. If a `Loader` is given then
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
196 `strict` does not apply.
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
197
135
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
198 """
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
199 if Loader is None:
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
200 Loader = config_loader_factory(strict=strict)
134
2f2e819e8d17 Check the return type of the JSON and YAML loading functions: they must be a dict alike
Franz Glasner <hg@dom66.de>
parents: 90
diff changeset
201 data = yaml.load(stream, Loader)
136
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
202 # Map an empty document to an empty dict
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
203 if data is None:
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
204 return DictImpl()
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
205 if not isinstance(data, DictImpl):
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
206 raise TypeError("YAML root object must be a mapping")
134
2f2e819e8d17 Check the return type of the JSON and YAML loading functions: they must be a dict alike
Franz Glasner <hg@dom66.de>
parents: 90
diff changeset
207 return data
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
208
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
209
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
210 def load_all(stream, Loader=None, strict=False):
135
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
211 """Parse the given `stream` and return a sequence of Python objects
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
212 corresponding to the documents in the `stream`.
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
213
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
214 If `strict` is `True` then duplicate mapping keys within a YAML
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
215 document are detected and prevented. If a `Loader` is given then
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
216 `strict` does not apply.
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
217
135
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
218 """
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
219 if Loader is None:
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
220 Loader = config_loader_factory(strict=strict)
134
2f2e819e8d17 Check the return type of the JSON and YAML loading functions: they must be a dict alike
Franz Glasner <hg@dom66.de>
parents: 90
diff changeset
221 data_all = yaml.load_all(stream, Loader)
136
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
222 rdata = []
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
223 for data in data_all:
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
224 if data is None:
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
225 rdata.append(DictImpl())
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
226 else:
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
227 if not isinstance(data, DictImpl):
134
2f2e819e8d17 Check the return type of the JSON and YAML loading functions: they must be a dict alike
Franz Glasner <hg@dom66.de>
parents: 90
diff changeset
228 raise TypeError("YAML root object must be a mapping")
136
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
229 rdata.append(data)
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
230 return rdata
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
231
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
232
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
233 def safe_load(stream, strict=False):
135
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
234 """Parse the given `stream` and return a Python object constructed
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
235 from for the first document in the stream.
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
236
136
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
237 Recognizes only standard YAML tags and cannot construct an
135
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
238 arbitrary Python object.
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
239
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
240 If `strict` is `True` then duplicate mapping keys within a YAML document
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
241 are detected and prevented.
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
242
135
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
243 """
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
244 data = yaml.load(stream,
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
245 Loader=config_safe_loader_factory(strict=strict))
136
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
246 # Map an empty document to an empty dict
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
247 if data is None:
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
248 return DictImpl()
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
249 if not isinstance(data, DictImpl):
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
250 raise TypeError("YAML root object must be a mapping")
134
2f2e819e8d17 Check the return type of the JSON and YAML loading functions: they must be a dict alike
Franz Glasner <hg@dom66.de>
parents: 90
diff changeset
251 return data
1
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
252
e4c63b4f1568 Provide a yaml wrapper that import with all-unicode strings on Python2 but does not path the Loader globally
Franz Glasner <f.glasner@feldmann-mg.com>
parents:
diff changeset
253
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
254 def safe_load_all(stream, strict=False):
135
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
255 """Return the list of all decoded YAML documents in the file `stream`.
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
256
136
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
257 Recognizes only standard YAML tags and cannot construct an
135
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
258 arbitrary Python object.
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
259
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
260 If `strict` is `True` then duplicate mapping keys within a YAML document
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
261 are detected and prevented.
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
262
135
b7b0cea8ec6e Document "configmix.yaml.loadXXX()" functions
Franz Glasner <hg@dom66.de>
parents: 134
diff changeset
263 """
291
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
264 data_all = yaml.load_all(stream,
edf5cc1ffd26 Provide an optional "strict" keyword flag to all YAML load functions to detect and prevent duplicate keys within a single YAML document
Franz Glasner <f.glasner@feldmann-mg.com>
parents: 250
diff changeset
265 Loader=config_safe_loader_factory(strict=strict))
136
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
266 rdata = []
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
267 for data in data_all:
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
268 if data is None:
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
269 rdata.append(DictImpl())
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
270 else:
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
271 if not isinstance(data, DictImpl):
134
2f2e819e8d17 Check the return type of the JSON and YAML loading functions: they must be a dict alike
Franz Glasner <hg@dom66.de>
parents: 90
diff changeset
272 raise TypeError("YAML root object must be a mapping")
136
eee1dd1f99bf Simplify the YAML return type check and map a "None" (empty document) result to an empty mapping
Franz Glasner <hg@dom66.de>
parents: 135
diff changeset
273 rdata.append(data)
134
2f2e819e8d17 Check the return type of the JSON and YAML loading functions: they must be a dict alike
Franz Glasner <hg@dom66.de>
parents: 90
diff changeset
274 return data_all