Mercurial > hgrepos > Python > libs > ConfigMix
diff docs/introduction.rst @ 215:ab3d0326419c
Doc: Move the "doc" to "docs"
| author | Franz Glasner <fzglas.hg@dom66.de> |
|---|---|
| date | Wed, 08 May 2019 09:23:37 +0200 |
| parents | doc/introduction.rst@fa660f084ceb |
| children | 57ff12610dc5 |
line wrap: on
line diff
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/docs/introduction.rst Wed May 08 09:23:37 2019 +0200 @@ -0,0 +1,402 @@ +.. -*- coding: utf-8; indent-tabs-mode: nil; -*- + +.. _introduction: + +Introduction +============ + +.. contents:: + :local: + +The configurations can be read from different types of files: + +- :ref:`YAML files <yaml-files>` +- :ref:`JSON files <json-files>` +- :ref:`INI files <ini-files>` +- :ref:`TOML files <toml-files>` +- :ref:`executable Python scripts <executable-python-scripts>` + + +.. _yaml-files: + +YAML Files +---------- + +Need the :mod:`yaml` package (https://github.com/yaml/pyyaml) +(e.g. ``pip install pyyaml``) + +.. note:: All strings are returned as Unicode text strings. + +.. note:: The root object must be a *mapping* and therefore decode + into a Python :class:`dict` alike. This is checked by the + implementation. + +An example is: + +.. literalinclude:: ../tests/data/conf10.yml + :language: yaml + + +.. _json-files: + +JSON files +---------- + +Read the JSON file with the help of Python's native :mod:`json` package. + +.. note:: All strings are returned as Unicode text strings. + +.. note:: The root object must be an *object* and therefore decode + into a Python :class:`dict` alike. This is checked by the + implementation. + +An example is: + +.. literalinclude:: ../tests/data/conf10.json + :language: js + +For comments in JSON files see section :ref:`comments`. + + +.. _ini-files: + +INI Files +--------- + +Read the file and all sections named in parameter `extract` are flattened +into the resulting dictionary. By default the section named ``config`` is +used. + +Normally all values are returned as Unicode text strings. +But values can be annotated and therefore interpreted as other types: + + ``:int:`` + The value is handled in the same way as a Python :class:`int` + literal + + ``:float:`` + The value is interpreted as :class:`float` + + ``:bool:`` + The resulting value is a :class:`bool` where + + ``1``, ``true``, ``yes``, ``on`` + yield a Python ``True`` + + ``0``, ``false``, ``no``, ``off`` + yield a Python ``False`` + + The evaluation is done *case-insensitively*. + +.. note:: All strings are returned as Unicode text strings. + +.. note:: Contrary to the behaviour of the standard Python :mod:`configparser` + module the INI file reader is *case-sensitive*. + +The example INI style configuration below yields an equivalent +configuration to the YAML configuration above: + +.. literalinclude:: ../tests/data/conf10.ini + :language: ini + +As can be seen in this example -- INI file internal value interpolation +is done as in Python's standard :mod:`configparser` module. + +This example also illustrates how INI sections are used to build a +tree-ish configuration dictionary. + + +.. _toml-files: + +TOML Files +---------- + +Read the TOML file with the help of the pure Python :mod:`toml` +package (https://github.com/uiri/toml) (e.g. ``pip install toml``). + +All TOML features map seamingless to "ConfigMix". + +The example TOML style configuration below yields an equivalent +configuration to the YAML configuration above: + + +.. literalinclude:: ../tests/data/conf10.toml + :language: ini + + +.. _executable-python-scripts: + +Executable Python Scripts +------------------------- + +What will be exported: + +1. If loading is done with the `extract` parameter only the given keys are + extracted from the script. + +2. Otherwise it is checked if the scripts defines an ``__all__`` + sequence. If there is one it's contents are the keys to be + extracted. + +3. If there is no ``__all__`` object all names not starting with an + underscore ``_`` are found. + +This is analogous to as Python modules behave when importing them with +``from module import *``. + +.. note:: The Python configuration files are evaluated with ``exec`` and not + imported. + +The example configuration by Python script below yields an equivalent +configuration to the YAML configuration above: + +.. literalinclude:: ../tests/data/conf10.py + :language: python + + +.. _loading-and-merging: + +Loading and Merging +------------------- + +Basic usage of the API is as follows in this example:: + + import configmix + + # + # Note: With conf10 merging is rather pointless because the tree + # files # are really the same configuration. But is doesn't harm + # also here. + # + config = configmix.load("conf10.yml", "conf10.ini", "conf10.py") + + # Get a -- possibly interpolated -- configuration variable's value + value1 = config.getvar_s("key1") + + # Get a -- possibly interpolated -- variable from within the tree + value2 = config.getvar_s("tree1.tree2.key4") + + +By default filenames of the configuration files must have the extensions +(case-sensitivety depends on your OS): + + ``.ini`` + for INI configuration files + + ``.json`` + for JSON configuration files + + ``.py`` + for Python configuration files + + ``.toml`` + for TOML configuration file + + ``.yml`` or ``.yaml`` + for YAML configuration files + + +.. _getting-values: + +Getting configuration variables +------------------------------- + +Get a -- possibly interpolated -- configuration variable's value with:: + + value1 = config.getvar_s("key1") + +Get a raw configuration variable's value with:: + + value1_raw = config.getvar("key1") + +Because the configuration is not only a plain list of but a tree of +key-value pairs you will want to fetch them by separating the individual +level keys with a point ``.``. + +Looking at the example in chapter :ref:`yaml-files` -- when calling +``config.getvar_s("tree1.tree2.key4")`` you will get the value +``get this as `tree1.tree2.key4'``. + +This is true for both methods :py:meth:`.Configuration.getvar` and +:py:meth:`.Configuration.getvar_s`. + +Both methods also perform :ref:`variable-interpolation` and handle +:ref:`variable-namespaces`. Filtering is not supported. So -- the +variable name arguments of :py:meth:`.Configuration.getvar` and +:py:meth:`.Configuration.getvar_s` are of the form +``[namespace:]variable``. + + +.. _comments: + +Comments +-------- + +By default all keys beginning with ``__comment`` or ``__doc`` are +filtered out and not given to the application. This allows comments in +JSON files -- but is not restricted to JSON files only. + +For all types of configuration files their respective standard comments +are allowed too. + + +.. _variable-namespaces: + +Variable Namespaces +------------------- + +Currently there are 4 namespaces: + +1. The unnamed namespace (which is also default). + + All the configuration variables are part of this namespace. + +2. The namespace ``OS`` + + Available functions: + + ``cwd`` + Contains the current working directory of the process + +3. The namespace ``ENV`` + + This namespace contains all the environment variables as they are + available from :py:data:`os.environ`. + +4. The namespace ``PY`` + + Contains selected values from the running Python: + + ``version`` + The return value of :py:func:`platform.python_version` + + ``version_maj_min`` + Just the major and minor version of the running Python + (``.`` separated) + + ``version_maj`` + Just the major version of the running Python + + ``implementation`` + The return value of :py:func:`platform.python_implementation` + + +Examples +~~~~~~~~ + +:: + + config.getvar("OS:cwd") + +yields the current working directory as :py:func:`os.getcwd` does. + + +.. _variable-interpolation: + +Variable Interpolation +---------------------- + +Configuration variable values that are read with +:py:meth:`.Configuration.getvar_s` are subject to variable +interpolation. The general syntactic pattern for this is:: + + {{[namespace:]variable[|filter[|filter...]]}} + +I.e.: between double curly braces an optional `namespace` name followed by +a colon ``:``, the `variable` and then zero or more filters, each one +introduced by a pipe symbol ``|``. + +Variables are expanded *lately* at runtime -- exactly when calling +:py:meth:`.Configuration.getvar_s`, +:py:meth:`.Configuration.substitute_variables_in_obj` or +:py:meth:`.Configuration.expand_variable` + + +Filter functions +~~~~~~~~~~~~~~~~ + +Interpolated values can be processed through a series of filter functions:: + + {{my.variable|filter1|filter2}} + +Available filter functions are: + + ``urlquote`` + + ``saslprep`` + + ``normpath`` + + ``abspath`` + + ``posixpath`` + + ``lower`` + + ``upper`` + + +Examples +~~~~~~~~ + +:: + + {{OS:cwd|posixpath}} + +expands to the current working directory as POSIX path: on Windows all +backslashes are replaced by forward slashes. + +:: + + {{ENV:PATH}} + +expands to the current search path from the process environment. + +:: + + {{PY:version}} + +expands to the current running Python version (e.g. ``3.6.4``). + +:: + + {{PY::implementation|upper}} + +expands to something like ``CPYTHON`` when using the standard Python +interpreter written in C. + + +Custom filename extensions and custom loaders +--------------------------------------------- + +If you want to have custom configuration file extensions and/or custom loaders +for custom configuration files you have various possibilities: + + Associate an additional new extension (e.g. ".conf") with an + existing configuration file style (e.g. YAML):: + + configmix.set_assoc("*.conf", configmix.get_assoc("*.yml")) + + Allow only files with extension ".cfg" in INI-style -- using the default + loader for INI-files:: + + configmix.clear_assoc() + configmix.set_assoc("*.cfg", configmix.get_default_assoc("*.ini")) + + Only a new configuration file style:: + + def my_custom_loader(filename): + ... + return some_dict_alike + + configmix.mode_loaders["myconfmode"] = my_custom_loader + configmix.clear_assoc() + configmix.set_assoc("*.my.configuration", "myconfmode") + + If :py:func:`~configmix.clear_assoc` will not be called then just a *new* + configuration file style will be installed. + + To select the loader not by extension but by an Emacs-compatible mode + declaration (e.g. ``mode: yaml``) in the first two lines of a file use:: + + configmix.set_assoc("*", configmix.try_determine_filemode)
