.. -*- coding: utf-8; indent-tabs-mode: nil; -*-

.. _introduction:

Introduction
============

.. contents::
   :local:

The configurations can be read from different types of files:

- :ref:`YAML files <yaml-files>`
- :ref:`JSON files <json-files>`
- :ref:`INI files <ini-files>`
- :ref:`TOML files <toml-files>`
- :ref:`executable Python scripts <executable-python-scripts>`


.. _yaml-files:

YAML Files
----------

Need the :mod:`yaml` package (https://github.com/yaml/pyyaml)
(e.g. ``pip install pyyaml``)

.. note:: All strings are returned as Unicode text strings.

.. note:: The root object must be a *mapping* and therefore decode
          into a Python :class:`dict` alike. This is checked by the
          implementation.

An example is:

.. literalinclude:: ../tests/data/conf10.yml
   :language: yaml


.. _json-files:

JSON files
----------

Read the JSON file with the help of Python's native :mod:`json` package.

.. note:: All strings are returned as Unicode text strings.

.. note:: The root object must be an *object* and therefore decode
          into a Python :class:`dict` alike. This is checked by the
          implementation.

An example is:

.. literalinclude:: ../tests/data/conf10.json
   :language: js

For comments in JSON files see section :ref:`comments`.


.. _ini-files:

INI Files
---------

Read the file and all sections named in parameter `extract` are flattened
into the resulting dictionary. By default the section named ``config`` is
used.

Normally all values are returned as Unicode text strings.
But values can be annotated and therefore interpreted as other types:

  ``:int:``
      The value is handled in the same way as a Python :class:`int`
      literal

  ``:float:``
      The value is interpreted as :class:`float`

  ``:bool:``
      The resulting value is a :class:`bool` where

        ``1``, ``true``, ``yes``, ``on``
           yield a Python ``True``

        ``0``, ``false``, ``no``, ``off``
           yield a Python ``False``

      The evaluation is done *case-insensitively*.

.. note:: All strings are returned as Unicode text strings.

.. note:: Contrary to the behaviour of the standard Python :mod:`configparser`
          module the INI file reader is *case-sensitive*.

The example INI style configuration below yields an equivalent
configuration to the YAML configuration above:

.. literalinclude:: ../tests/data/conf10.ini
   :language: ini

As can be seen in this example -- INI file internal value interpolation
is done as in Python's standard :mod:`configparser` module.

This example also illustrates how INI sections are used to build a
tree-ish configuration dictionary.


.. _toml-files:

TOML Files
----------

Read the TOML file with the help of the pure Python :mod:`toml`
package (https://github.com/uiri/toml) (e.g. ``pip install toml``).

All TOML features map seamingless to "ConfigMix".

The example TOML style configuration below yields an equivalent
configuration to the YAML configuration above:


.. literalinclude:: ../tests/data/conf10.toml
   :language: ini


.. _executable-python-scripts:

Executable Python Scripts
-------------------------

What will be exported:

1. If loading is done with the `extract` parameter only the given keys are
   extracted from the script.

2. Otherwise it is checked if the scripts defines an ``__all__``
   sequence. If there is one it's contents are the keys to be
   extracted.

3. If there is no ``__all__`` object all names not starting with an
   underscore ``_`` are found.

This is analogous to as Python modules behave when importing them with
``from module import *``.

.. note:: The Python configuration files are evaluated with ``exec`` and not
          imported.

The example configuration by Python script below yields an equivalent
configuration to the YAML configuration above:

.. literalinclude:: ../tests/data/conf10.py
   :language: python


.. _loading-and-merging:

Loading and Merging
-------------------

Basic usage of the API is as follows in this example::

    import configmix

    #
    # Note: With conf10 merging is rather pointless because the tree
    # files # are really the same configuration. But is doesn't harm
    # also here.
    #
    config = configmix.load("conf10.yml", "conf10.ini", "conf10.py")

    # Get a -- possibly interpolated -- configuration variable's value
    value1 = config.getvar_s("key1")

    # Get a -- possibly interpolated -- variable from within the tree
    value2 = config.getvar_s("tree1.tree2.key4")


By default filenames of the configuration files must have the extensions
(case-sensitivety depends on your OS):

  ``.ini``
    for INI configuration files

  ``.json``
    for JSON configuration files

  ``.py``
    for Python configuration files

  ``.toml``
    for TOML configuration file

  ``.yml`` or ``.yaml``
    for YAML configuration files


.. _getting-values:

Getting configuration variables
-------------------------------

Get a -- possibly interpolated -- configuration variable's value with::

    value1 = config.getvar_s("key1")

Get a raw configuration variable's value with::

    value1_raw = config.getvar("key1")

Because the configuration is not only a plain list of but a tree of
key-value pairs you will want to fetch them by separating the individual
level keys with a point ``.``.

Looking at the example in chapter :ref:`yaml-files` -- when calling
``config.getvar_s("tree1.tree2.key4")`` you will get the value
``get this as `tree1.tree2.key4'``.

This is true for both methods :py:meth:`.Configuration.getvar` and
:py:meth:`.Configuration.getvar_s`.

Both methods also perform :ref:`variable-interpolation` and handle
:ref:`variable-namespaces`. Filtering is not supported. So -- the
variable name arguments of :py:meth:`.Configuration.getvar` and
:py:meth:`.Configuration.getvar_s` are of the form
``[namespace:]variable``.


.. _merging-deletions:

Deletions
---------

By using the special value ``{{::DEL::}}`` the corresponding key-value
pair is deleted when merging is done.


.. _comments:

Comments
--------

By default all keys beginning with ``__comment`` or ``__doc`` are
filtered out and not given to the application. This allows comments in
JSON files -- but is not restricted to JSON files only.

For all types of configuration files their respective standard comments
are allowed too.


.. _variable-namespaces:

Variable Namespaces
-------------------

Currently there are 4 namespaces:

1. The unnamed namespace (which is also default).

   All the configuration variables are part of this namespace.

2. The namespace ``OS``

   Available functions:

     ``cwd``
         Contains the current working directory of the process

     ``node``
         Contains the current node's computername (or whatever
         :py:func:`platform.node` returns)

3. The namespace ``ENV``

   This namespace contains all the environment variables as they are
   available from :py:data:`os.environ`.

4. The namespace ``PY``

   Contains selected values from the running Python:

     ``version``
         The return value of :py:func:`platform.python_version`

     ``version_maj_min``
         Just the major and minor version of the running Python
         (``.`` separated)

     ``version_maj``
         Just the major version of the running Python

     ``implementation``
         The return value of :py:func:`platform.python_implementation`

5. The namespace ``AWS``

   Contains some metadata for AWS instances when running from within
   AWS:

     ``metadata.instance-id``

     ``metadata.placement.region``

     ``metadata.placement.availability-zone``

     ``dynamic.instance-identity.region``
       and all other properties of the instance-identity document
       (e.g. ``instanceId``, ``instanceType``, ``imageId``, ``pendingTime``,
       ``architecture``, ``availabilityZone``, ``privateIp``, ``version``
       et al.).


Examples
~~~~~~~~

::

     config.getvar("OS:cwd")

yields the current working directory as :py:func:`os.getcwd` does.


.. _variable-interpolation:

Variable Interpolation
----------------------

Configuration variable values that are read with
:py:meth:`.Configuration.getvar_s` are subject to variable
interpolation. The general syntactic pattern for this is::

    {{[namespace:]variable[|filter[|filter...]]}}

I.e.: between double curly braces an optional `namespace` name followed by
a colon ``:``, the `variable` and then zero or more filters, each one
introduced by a pipe symbol ``|``.

Variables are expanded *lately* at runtime -- exactly when calling
:py:meth:`.Configuration.getvar_s`,
:py:meth:`.Configuration.substitute_variables_in_obj` or
:py:meth:`.Configuration.expand_variable`


Filter functions
~~~~~~~~~~~~~~~~

Interpolated values can be processed through a series of filter functions::

    {{my.variable|filter1|filter2}}

Available filter functions are:

  ``urlquote``

  ``saslprep``

  ``normpath``

  ``abspath``

  ``posixpath``

  ``lower``

  ``upper``


Examples
~~~~~~~~

::

    {{OS:cwd|posixpath}}

expands to the current working directory as POSIX path: on Windows all
backslashes are replaced by forward slashes.

::

    {{ENV:PATH}}

expands to the current search path from the process environment.

::

    {{PY:version}}

expands to the current running Python version (e.g. ``3.6.4``).

::

    {{PY::implementation|upper}}

expands to something like ``CPYTHON`` when using the standard Python
interpreter written in C.


Custom filename extensions and custom loaders
---------------------------------------------

If you want to have custom configuration file extensions and/or custom loaders
for custom configuration files you have various possibilities:

  Associate an additional new extension (e.g. ".conf") with an
  existing configuration file style (e.g. YAML)::

    configmix.set_assoc("*.conf", configmix.get_assoc("*.yml"))

  Allow only files with extension ".cfg" in INI-style -- using the default
  loader for INI-files::

    configmix.clear_assoc()
    configmix.set_assoc("*.cfg", configmix.get_default_assoc("*.ini"))

  Only a new configuration file style::

    def my_custom_loader(filename):
        ...
        return some_dict_alike

    configmix.mode_loaders["myconfmode"] = my_custom_loader
    configmix.clear_assoc()
    configmix.set_assoc("*.my.configuration", "myconfmode")

  If :py:func:`~configmix.clear_assoc` will not be called then just a *new*
  configuration file style will be installed.

  To select the loader not by extension but by an Emacs-compatible mode
  declaration (e.g. ``mode: yaml``)  in the first two lines of a file use::

    configmix.set_assoc("*", configmix.try_determine_filemode)