diff docs/introduction.rst @ 215:ab3d0326419c

Doc: Move the "doc" to "docs"
author Franz Glasner <fzglas.hg@dom66.de>
date Wed, 08 May 2019 09:23:37 +0200
parents doc/introduction.rst@fa660f084ceb
children 57ff12610dc5
line wrap: on
line diff
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/docs/introduction.rst	Wed May 08 09:23:37 2019 +0200
@@ -0,0 +1,402 @@
+.. -*- coding: utf-8; indent-tabs-mode: nil; -*-
+
+.. _introduction:
+
+Introduction
+============
+
+.. contents::
+   :local:
+
+The configurations can be read from different types of files:
+
+- :ref:`YAML files <yaml-files>`
+- :ref:`JSON files <json-files>`
+- :ref:`INI files <ini-files>`
+- :ref:`TOML files <toml-files>`
+- :ref:`executable Python scripts <executable-python-scripts>`
+
+
+.. _yaml-files:
+
+YAML Files
+----------
+
+Need the :mod:`yaml` package (https://github.com/yaml/pyyaml)
+(e.g. ``pip install pyyaml``)
+
+.. note:: All strings are returned as Unicode text strings.
+
+.. note:: The root object must be a *mapping* and therefore decode
+          into a Python :class:`dict` alike. This is checked by the
+          implementation.
+
+An example is:
+
+.. literalinclude:: ../tests/data/conf10.yml
+   :language: yaml
+
+
+.. _json-files:
+
+JSON files
+----------
+
+Read the JSON file with the help of Python's native :mod:`json` package.
+
+.. note:: All strings are returned as Unicode text strings.
+
+.. note:: The root object must be an *object* and therefore decode
+          into a Python :class:`dict` alike. This is checked by the
+          implementation.
+
+An example is:
+
+.. literalinclude:: ../tests/data/conf10.json
+   :language: js
+
+For comments in JSON files see section :ref:`comments`.
+
+
+.. _ini-files:
+
+INI Files
+---------
+
+Read the file and all sections named in parameter `extract` are flattened
+into the resulting dictionary. By default the section named ``config`` is
+used.
+
+Normally all values are returned as Unicode text strings.
+But values can be annotated and therefore interpreted as other types:
+
+  ``:int:``
+      The value is handled in the same way as a Python :class:`int`
+      literal
+
+  ``:float:``
+      The value is interpreted as :class:`float`
+
+  ``:bool:``
+      The resulting value is a :class:`bool` where
+
+        ``1``, ``true``, ``yes``, ``on``
+           yield a Python ``True``
+
+        ``0``, ``false``, ``no``, ``off``
+           yield a Python ``False``
+
+      The evaluation is done *case-insensitively*.
+
+.. note:: All strings are returned as Unicode text strings.
+
+.. note:: Contrary to the behaviour of the standard Python :mod:`configparser`
+          module the INI file reader is *case-sensitive*.
+
+The example INI style configuration below yields an equivalent
+configuration to the YAML configuration above:
+
+.. literalinclude:: ../tests/data/conf10.ini
+   :language: ini
+
+As can be seen in this example -- INI file internal value interpolation
+is done as in Python's standard :mod:`configparser` module.
+
+This example also illustrates how INI sections are used to build a
+tree-ish configuration dictionary.
+
+
+.. _toml-files:
+
+TOML Files
+----------
+
+Read the TOML file with the help of the pure Python :mod:`toml`
+package (https://github.com/uiri/toml) (e.g. ``pip install toml``).
+
+All TOML features map seamingless to "ConfigMix".
+
+The example TOML style configuration below yields an equivalent
+configuration to the YAML configuration above:
+
+
+.. literalinclude:: ../tests/data/conf10.toml
+   :language: ini
+
+
+.. _executable-python-scripts:
+
+Executable Python Scripts
+-------------------------
+
+What will be exported:
+
+1. If loading is done with the `extract` parameter only the given keys are
+   extracted from the script.
+
+2. Otherwise it is checked if the scripts defines an ``__all__``
+   sequence. If there is one it's contents are the keys to be
+   extracted.
+
+3. If there is no ``__all__`` object all names not starting with an
+   underscore ``_`` are found.
+
+This is analogous to as Python modules behave when importing them with
+``from module import *``.
+
+.. note:: The Python configuration files are evaluated with ``exec`` and not
+          imported.
+
+The example configuration by Python script below yields an equivalent
+configuration to the YAML configuration above:
+
+.. literalinclude:: ../tests/data/conf10.py
+   :language: python
+
+
+.. _loading-and-merging:
+
+Loading and Merging
+-------------------
+
+Basic usage of the API is as follows in this example::
+
+    import configmix
+
+    #
+    # Note: With conf10 merging is rather pointless because the tree
+    # files # are really the same configuration. But is doesn't harm
+    # also here.
+    #
+    config = configmix.load("conf10.yml", "conf10.ini", "conf10.py")
+
+    # Get a -- possibly interpolated -- configuration variable's value
+    value1 = config.getvar_s("key1")
+
+    # Get a -- possibly interpolated -- variable from within the tree
+    value2 = config.getvar_s("tree1.tree2.key4")
+
+
+By default filenames of the configuration files must have the extensions
+(case-sensitivety depends on your OS):
+
+  ``.ini``
+    for INI configuration files
+
+  ``.json``
+    for JSON configuration files
+
+  ``.py``
+    for Python configuration files
+
+  ``.toml``
+    for TOML configuration file
+
+  ``.yml`` or ``.yaml``
+    for YAML configuration files
+
+
+.. _getting-values:
+
+Getting configuration variables
+-------------------------------
+
+Get a -- possibly interpolated -- configuration variable's value with::
+
+    value1 = config.getvar_s("key1")
+
+Get a raw configuration variable's value with::
+
+    value1_raw = config.getvar("key1")
+
+Because the configuration is not only a plain list of but a tree of
+key-value pairs you will want to fetch them by separating the individual
+level keys with a point ``.``.
+
+Looking at the example in chapter :ref:`yaml-files` -- when calling
+``config.getvar_s("tree1.tree2.key4")`` you will get the value
+``get this as `tree1.tree2.key4'``.
+
+This is true for both methods :py:meth:`.Configuration.getvar` and
+:py:meth:`.Configuration.getvar_s`.
+
+Both methods also perform :ref:`variable-interpolation` and handle
+:ref:`variable-namespaces`. Filtering is not supported. So -- the
+variable name arguments of :py:meth:`.Configuration.getvar` and
+:py:meth:`.Configuration.getvar_s` are of the form
+``[namespace:]variable``.
+
+
+.. _comments:
+
+Comments
+--------
+
+By default all keys beginning with ``__comment`` or ``__doc`` are
+filtered out and not given to the application. This allows comments in
+JSON files -- but is not restricted to JSON files only.
+
+For all types of configuration files their respective standard comments
+are allowed too.
+
+
+.. _variable-namespaces:
+
+Variable Namespaces
+-------------------
+
+Currently there are 4 namespaces:
+
+1. The unnamed namespace (which is also default).
+
+   All the configuration variables are part of this namespace.
+
+2. The namespace ``OS``
+
+   Available functions:
+
+     ``cwd``
+         Contains the current working directory of the process
+
+3. The namespace ``ENV``
+
+   This namespace contains all the environment variables as they are
+   available from :py:data:`os.environ`.
+
+4. The namespace ``PY``
+
+   Contains selected values from the running Python:
+
+     ``version``
+         The return value of :py:func:`platform.python_version`
+
+     ``version_maj_min``
+         Just the major and minor version of the running Python
+         (``.`` separated)
+
+     ``version_maj``
+         Just the major version of the running Python
+
+     ``implementation``
+         The return value of :py:func:`platform.python_implementation`
+
+
+Examples
+~~~~~~~~
+
+::
+
+     config.getvar("OS:cwd")
+
+yields the current working directory as :py:func:`os.getcwd` does.
+
+
+.. _variable-interpolation:
+
+Variable Interpolation
+----------------------
+
+Configuration variable values that are read with
+:py:meth:`.Configuration.getvar_s` are subject to variable
+interpolation. The general syntactic pattern for this is::
+
+    {{[namespace:]variable[|filter[|filter...]]}}
+
+I.e.: between double curly braces an optional `namespace` name followed by
+a colon ``:``, the `variable` and then zero or more filters, each one
+introduced by a pipe symbol ``|``.
+
+Variables are expanded *lately* at runtime -- exactly when calling
+:py:meth:`.Configuration.getvar_s`,
+:py:meth:`.Configuration.substitute_variables_in_obj` or
+:py:meth:`.Configuration.expand_variable`
+
+
+Filter functions
+~~~~~~~~~~~~~~~~
+
+Interpolated values can be processed through a series of filter functions::
+
+    {{my.variable|filter1|filter2}}
+
+Available filter functions are:
+
+  ``urlquote``
+
+  ``saslprep``
+
+  ``normpath``
+
+  ``abspath``
+
+  ``posixpath``
+
+  ``lower``
+
+  ``upper``
+
+
+Examples
+~~~~~~~~
+
+::
+
+    {{OS:cwd|posixpath}}
+
+expands to the current working directory as POSIX path: on Windows all
+backslashes are replaced by forward slashes.
+
+::
+
+    {{ENV:PATH}}
+
+expands to the current search path from the process environment.
+
+::
+
+    {{PY:version}}
+
+expands to the current running Python version (e.g. ``3.6.4``).
+
+::
+
+    {{PY::implementation|upper}}
+
+expands to something like ``CPYTHON`` when using the standard Python
+interpreter written in C.
+
+
+Custom filename extensions and custom loaders
+---------------------------------------------
+
+If you want to have custom configuration file extensions and/or custom loaders
+for custom configuration files you have various possibilities:
+
+  Associate an additional new extension (e.g. ".conf") with an
+  existing configuration file style (e.g. YAML)::
+
+    configmix.set_assoc("*.conf", configmix.get_assoc("*.yml"))
+
+  Allow only files with extension ".cfg" in INI-style -- using the default
+  loader for INI-files::
+
+    configmix.clear_assoc()
+    configmix.set_assoc("*.cfg", configmix.get_default_assoc("*.ini"))
+
+  Only a new configuration file style::
+
+    def my_custom_loader(filename):
+        ...
+        return some_dict_alike
+
+    configmix.mode_loaders["myconfmode"] = my_custom_loader
+    configmix.clear_assoc()
+    configmix.set_assoc("*.my.configuration", "myconfmode")
+
+  If :py:func:`~configmix.clear_assoc` will not be called then just a *new*
+  configuration file style will be installed.
+
+  To select the loader not by extension but by an Emacs-compatible mode
+  declaration (e.g. ``mode: yaml``)  in the first two lines of a file use::
+
+    configmix.set_assoc("*", configmix.try_determine_filemode)