Merge lp:~albertog/siesta/4.1-yaml into lp:siesta/4.1

[Alberto Garcia]

  YAML infrastructure for output testing

  (By Yann Pouillon)

  The implementation consists of:

  * A simple hook in siesta.F to call the siesta_write_yaml routine
  in module Src/m_io_yaml.F90.

  * A YAML reference directory: Tests/YAML_Refs/
    (currently populated by just three token examples)

  * A post-processing script: Tests/Scripts/yaml_compare.py

  * A configuration file Tests/siesta-testsuite.yml

  See Docs/yaml_tests/README for more information.

  Notes

  - Only energies are processed in this initial implementation.
  - The post-processing script depends on ruamel.yaml (pip install [--local] ruamel.yaml)
  - SIESTA does not require LibYAML to emit YAML files

[Yann Pouillon (pouillon)]

I would not have included the Docs/yaml_tests/_build/ subdirectory, since it is generated by Sphinx. If it was in the Bazaar branch, it's probably because I imported it for the demo I made at Simune's office. The important files are the *.rst found in Docs/yaml_tests/. The Makefile will do the rest, provided that Sphinx is installed. If not, the .rst files remain highly readable.

[lp:~albertog/siesta/4.1-yaml]

922. By Alberto Garcia on 2018-05-29

Remove _build/html files in Docs/yaml_tests

[Alberto Garcia (albertog)]

OK. Removed the _build/html directory.

[Yann Pouillon (pouillon)]

Then you're good to go. Thanks!

review: Approve

[Nick Papior (nickpapior)]

Great job! I have two changes.

The transiesta flag in 4.1 is now intrinsic. So no need to do pre-processor statement checks there.

[lp:~albertog/siesta/4.1-yaml]

923. By Alberto Garcia on 2018-05-30

Remove Transiesta pre-processed items

=== added file 'Docs/YAML.README'
--- Docs/YAML.README	1970-01-01 00:00:00 +0000
+++ Docs/YAML.README	2018-05-30 08:59:05 +0000
@@ -0,0 +1,13 @@
+One YAML Format Version: 1.2
+One Fortran module: Src/m_io_yaml.F90
+One I/O routine: siesta_write_yaml (no arguments)
+One call from the main SIESTA routine: Src/siesta.F:104
+One YAML reference directory: Tests/YAML_Refs/
+One post-processing script: Tests/Scripts/yaml_compare.py
+One global report with detailed info: Tests/tests-report.yml
+
+Notes
+
+The post-processing script depends on ruamel.yaml pip install [--local] ruamel.yaml
+
+SIESTA does not require LibYAML to emit YAML files

=== added directory 'Docs/yaml_tests'
=== added file 'Docs/yaml_tests/Makefile'
--- Docs/yaml_tests/Makefile	1970-01-01 00:00:00 +0000
+++ Docs/yaml_tests/Makefile	2018-05-30 08:59:05 +0000
@@ -0,0 +1,216 @@
+# Makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line.
+SPHINXOPTS    =
+SPHINXBUILD   = sphinx-build
+PAPER         =
+BUILDDIR      = _build
+
+# User-friendly check for sphinx-build
+ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1)
+$(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/)
+endif
+
+# Internal variables.
+PAPEROPT_a4     = -D latex_paper_size=a4
+PAPEROPT_letter = -D latex_paper_size=letter
+ALLSPHINXOPTS   = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
+# the i18n builder cannot share the environment and doctrees with the others
+I18NSPHINXOPTS  = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
+
+.PHONY: help
+help:
+	@echo "Please use \`make <target>' where <target> is one of"
+	@echo "  html       to make standalone HTML files"
+	@echo "  dirhtml    to make HTML files named index.html in directories"
+	@echo "  singlehtml to make a single large HTML file"
+	@echo "  pickle     to make pickle files"
+	@echo "  json       to make JSON files"
+	@echo "  htmlhelp   to make HTML files and a HTML help project"
+	@echo "  qthelp     to make HTML files and a qthelp project"
+	@echo "  applehelp  to make an Apple Help Book"
+	@echo "  devhelp    to make HTML files and a Devhelp project"
+	@echo "  epub       to make an epub"
+	@echo "  latex      to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
+	@echo "  latexpdf   to make LaTeX files and run them through pdflatex"
+	@echo "  latexpdfja to make LaTeX files and run them through platex/dvipdfmx"
+	@echo "  text       to make text files"
+	@echo "  man        to make manual pages"
+	@echo "  texinfo    to make Texinfo files"
+	@echo "  info       to make Texinfo files and run them through makeinfo"
+	@echo "  gettext    to make PO message catalogs"
+	@echo "  changes    to make an overview of all changed/added/deprecated items"
+	@echo "  xml        to make Docutils-native XML files"
+	@echo "  pseudoxml  to make pseudoxml-XML files for display purposes"
+	@echo "  linkcheck  to check all external links for integrity"
+	@echo "  doctest    to run all doctests embedded in the documentation (if enabled)"
+	@echo "  coverage   to run coverage check of the documentation (if enabled)"
+
+.PHONY: clean
+clean:
+	rm -rf $(BUILDDIR)/*
+
+.PHONY: html
+html:
+	$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
+	@echo
+	@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
+
+.PHONY: dirhtml
+dirhtml:
+	$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
+	@echo
+	@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
+
+.PHONY: singlehtml
+singlehtml:
+	$(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml
+	@echo
+	@echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml."
+
+.PHONY: pickle
+pickle:
+	$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
+	@echo
+	@echo "Build finished; now you can process the pickle files."
+
+.PHONY: json
+json:
+	$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
+	@echo
+	@echo "Build finished; now you can process the JSON files."
+
+.PHONY: htmlhelp
+htmlhelp:
+	$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp
+	@echo
+	@echo "Build finished; now you can run HTML Help Workshop with the" \
+	      ".hhp project file in $(BUILDDIR)/htmlhelp."
+
+.PHONY: qthelp
+qthelp:
+	$(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp
+	@echo
+	@echo "Build finished; now you can run "qcollectiongenerator" with the" \
+	      ".qhcp project file in $(BUILDDIR)/qthelp, like this:"
+	@echo "# qcollectiongenerator $(BUILDDIR)/qthelp/YAMLTestsinSIESTA.qhcp"
+	@echo "To view the help file:"
+	@echo "# assistant -collectionFile $(BUILDDIR)/qthelp/YAMLTestsinSIESTA.qhc"
+
+.PHONY: applehelp
+applehelp:
+	$(SPHINXBUILD) -b applehelp $(ALLSPHINXOPTS) $(BUILDDIR)/applehelp
+	@echo
+	@echo "Build finished. The help book is in $(BUILDDIR)/applehelp."
+	@echo "N.B. You won't be able to view it unless you put it in" \
+	      "~/Library/Documentation/Help or install it in your application" \
+	      "bundle."
+
+.PHONY: devhelp
+devhelp:
+	$(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp
+	@echo
+	@echo "Build finished."
+	@echo "To view the help file:"
+	@echo "# mkdir -p $$HOME/.local/share/devhelp/YAMLTestsinSIESTA"
+	@echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/YAMLTestsinSIESTA"
+	@echo "# devhelp"
+
+.PHONY: epub
+epub:
+	$(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub
+	@echo
+	@echo "Build finished. The epub file is in $(BUILDDIR)/epub."
+
+.PHONY: latex
+latex:
+	$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
+	@echo
+	@echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
+	@echo "Run \`make' in that directory to run these through (pdf)latex" \
+	      "(use \`make latexpdf' here to do that automatically)."
+
+.PHONY: latexpdf
+latexpdf:
+	$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
+	@echo "Running LaTeX files through pdflatex..."
+	$(MAKE) -C $(BUILDDIR)/latex all-pdf
+	@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
+
+.PHONY: latexpdfja
+latexpdfja:
+	$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
+	@echo "Running LaTeX files through platex and dvipdfmx..."
+	$(MAKE) -C $(BUILDDIR)/latex all-pdf-ja
+	@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
+
+.PHONY: text
+text:
+	$(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text
+	@echo
+	@echo "Build finished. The text files are in $(BUILDDIR)/text."
+
+.PHONY: man
+man:
+	$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man
+	@echo
+	@echo "Build finished. The manual pages are in $(BUILDDIR)/man."
+
+.PHONY: texinfo
+texinfo:
+	$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
+	@echo
+	@echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo."
+	@echo "Run \`make' in that directory to run these through makeinfo" \
+	      "(use \`make info' here to do that automatically)."
+
+.PHONY: info
+info:
+	$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
+	@echo "Running Texinfo files through makeinfo..."
+	make -C $(BUILDDIR)/texinfo info
+	@echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo."
+
+.PHONY: gettext
+gettext:
+	$(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale
+	@echo
+	@echo "Build finished. The message catalogs are in $(BUILDDIR)/locale."
+
+.PHONY: changes
+changes:
+	$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
+	@echo
+	@echo "The overview file is in $(BUILDDIR)/changes."
+
+.PHONY: linkcheck
+linkcheck:
+	$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
+	@echo
+	@echo "Link check complete; look for any errors in the above output " \
+	      "or in $(BUILDDIR)/linkcheck/output.txt."
+
+.PHONY: doctest
+doctest:
+	$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
+	@echo "Testing of doctests in the sources finished, look at the " \
+	      "results in $(BUILDDIR)/doctest/output.txt."
+
+.PHONY: coverage
+coverage:
+	$(SPHINXBUILD) -b coverage $(ALLSPHINXOPTS) $(BUILDDIR)/coverage
+	@echo "Testing of coverage in the sources finished, look at the " \
+	      "results in $(BUILDDIR)/coverage/python.txt."
+
+.PHONY: xml
+xml:
+	$(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml
+	@echo
+	@echo "Build finished. The XML files are in $(BUILDDIR)/xml."
+
+.PHONY: pseudoxml
+pseudoxml:
+	$(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml
+	@echo
+	@echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml."

=== added file 'Docs/yaml_tests/README'
--- Docs/yaml_tests/README	1970-01-01 00:00:00 +0000
+++ Docs/yaml_tests/README	2018-05-30 08:59:05 +0000
@@ -0,0 +1,5 @@
+You need sphinx to re-create the documentation in html or other formats.
+(Type 'make help' for the various options)
+
+The source .rst files themselves are quite readable.
+

=== added directory 'Docs/yaml_tests/_static'
=== added file 'Docs/yaml_tests/analysis.rst'
--- Docs/yaml_tests/analysis.rst	1970-01-01 00:00:00 +0000
+++ Docs/yaml_tests/analysis.rst	2018-05-30 08:59:05 +0000
@@ -0,0 +1,137 @@
+===============================================
+Analyzing tests using the YAML output of SIESTA
+===============================================
+
+Minimum requirements
+--------------------
+
+Performing the analysis of the YAML data produced by SIESTA requires the
+following packages:
+
+- Python 3.4 or later;
+- The *ruamel.yaml* Python package.
+
+Optionally, the *termcolor* Python package can be installed to provide
+colorized on-screen output (recommended).
+
+To install Python, please refer to the documentation of your system
+distribution. Once Python is installed, you can install in turn the additional
+Python packages using PIP:
+
+.. code::
+
+   pip install ruamel.yaml termcolor
+
+Depending on how you have installed Python, the PIP executable may have a
+different name, e.g. *pip3* or *pip3.6*.
+
+
+Data workflow
+-------------
+
+The data of every test case goes through two steps: generation by a SIESTA
+executable and post-processing by a Python script. The workflow is open to
+further post-processing thanks to the generation of a YAML report of the
+analysis.
+
+The following diagram illustrates how the data is processed when running a
+YAML-enabled SIESTA executable:
+
+.. graphviz::
+
+   digraph yaml_workflow {
+
+     graph [fontname="Helvetica"];
+     node [fontname="Helvetica"];
+     edge [fontname="Helvetica"];
+
+     node [shape="box", style="filled", fillcolor="skyblue"];
+     siesta [label="SIESTA"];
+     yaml_cmp [label="yaml_compare.py"];
+
+     node [shape="oval", style="filled", fillcolor="gold"];
+     config [label="siesta-testsuite.yml"];
+     input [label="Input file", fillcolor="mistyrose"];
+     outvars [label="OUTVARS.yml"];
+     refs [label="YAML_Refs/"];
+     report [label="tests-report.yml"];
+
+     node [shape="octagon", style="filled", fillcolor="white"];
+     summary [label="Screen summary"];
+
+     input -> siesta;
+     siesta -> outvars;
+     config -> yaml_cmp;
+     outvars -> yaml_cmp;
+     refs -> yaml_cmp;
+     yaml_cmp -> report;
+     yaml_cmp -> summary;
+   }
+
+At the end of the calculation, SIESTA writes down a file named *OUTVARS.yml*,
+which contains information about both the build parameters of SIESTA and the
+values of selected variables (energies for the moment). The *yaml_compare.py*
+script then analyzes the content of *OUTVARS.yml* for each test case and
+compares it to the corresponding reference file located in the
+*Tests/YAML_Refs/* directory. The behaviour of the script can be tuned by
+editing the *siesta-testuite.yml* configuration file, which contains the full
+specifications of the SIESTA Test Suite, including fine-grained tolerances for
+the comparisons. Once the output of all tests has been processed, the
+*yaml_compare.py* script generates a detailed report in YAML format,
+*tests-report.yml*, as well as a condensed summary on the terminal.
+
+The YAML report can be read by humans but is optimized for further automated
+analysis and problem diagnosis, while the on-screen summary is intended
+primarily for the user and can be colorized if the terminal allows it (see the
+*--fancy* option of *yaml_compare.py*).
+
+
+How to use yaml_compare.py
+--------------------------
+
+The *yaml_compare.py* script is loacted in *Tests/Scripts/* and expected to
+run from the top directory of the SIESTA Test Suite (*Tests/*), through the
+following command: *./Scripts/yaml_compare.py [options]*. It goes through all
+test cases described in *siesta-testsuite.yml* and follows the specifications
+available therein.
+
+Several parameters of the scripts can be tuned through command-line options.
+For detailed information about them, just run:
+
+.. code::
+
+   ./Scripts/yaml_compare.py --help
+
+The *termcolor* Python package is necessary to get fancy colorized output
+through the *--fancy* option, which will simply be ignored otherwise. If this
+option is not used, the on-screen summary displayed by *yaml_compare.py* is
+valid RestructuredText and can be further processed to produce HTML and/or PDF
+summaries.
+
+
+Configuring the tests
+---------------------
+
+All configuration parameters related to the processing of the tests are stored
+in the self-documented *Tests/siesta-testsuite.yml* file. All information on
+how to tune the various parameters can thus be found in the file itself.
+
+
+Generating new references
+-------------------------
+
+The procedure to generate new references is the following:
+
+#. Build SIESTA using the reference configuration specifications.
+#. Run the test case and carefully check the output, both on screen and in
+   *OUTVARS.yml*.
+#. Copy *OUTVARS.yml* to *~siesta/Tests/YAML_Refs/test_name.yml*, where you
+   replace *test_name* by the actual name of the test.
+#. Add the test case specifications into *~siesta/Tests/siesta-testsuite.yml*.
+#. Run *yaml_compare.py* and check that the reference passes the test against
+   itself.
+
+.. note:: 
+
+   TODO: Define the reference specifications for the build of SIESTA.
+

=== added file 'Docs/yaml_tests/conf.py'
--- Docs/yaml_tests/conf.py	1970-01-01 00:00:00 +0000
+++ Docs/yaml_tests/conf.py	2018-05-30 08:59:05 +0000
@@ -0,0 +1,290 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+#
+# YAML Tests in SIESTA documentation build configuration file, created by
+# sphinx-quickstart on Thu Jan 18 15:13:04 2018.
+#
+# This file is execfile()d with the current directory set to its
+# containing dir.
+#
+# Note that not all possible configuration values are present in this
+# autogenerated file.
+#
+# All configuration values have a default; values that are commented out
+# serve to show the default.
+
+import sys
+import os
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#sys.path.insert(0, os.path.abspath('.'))
+
+# -- General configuration ------------------------------------------------
+
+# If your documentation needs a minimal Sphinx version, state it here.
+#needs_sphinx = '1.0'
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = [
+    'sphinx.ext.graphviz',
+    'sphinx.ext.mathjax',
+]
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# The suffix(es) of source filenames.
+# You can specify multiple suffix as a list of string:
+# source_suffix = ['.rst', '.md']
+source_suffix = '.rst'
+
+# The encoding of source files.
+#source_encoding = 'utf-8-sig'
+
+# The master toctree document.
+master_doc = 'index'
+
+# General information about the project.
+project = 'YAML Tests in SIESTA'
+copyright = '2018, Yann Pouillon'
+author = 'Yann Pouillon'
+
+# The version info for the project you're documenting, acts as replacement for
+# |version| and |release|, also used in various other places throughout the
+# built documents.
+#
+# The short X.Y version.
+version = '0.1'
+# The full version, including alpha/beta/rc tags.
+release = '0.1.0'
+
+# The language for content autogenerated by Sphinx. Refer to documentation
+# for a list of supported languages.
+#
+# This is also used if you do content translation via gettext catalogs.
+# Usually you set "language" from the command line for these cases.
+language = None
+
+# There are two options for replacing |today|: either, you set today to some
+# non-false value, then it is used:
+#today = ''
+# Else, today_fmt is used as the format for a strftime call.
+#today_fmt = '%B %d, %Y'
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+exclude_patterns = ['_build']
+
+# The reST default role (used for this markup: `text`) to use for all
+# documents.
+#default_role = None
+
+# If true, '()' will be appended to :func: etc. cross-reference text.
+#add_function_parentheses = True
+
+# If true, the current module name will be prepended to all description
+# unit titles (such as .. function::).
+#add_module_names = True
+
+# If true, sectionauthor and moduleauthor directives will be shown in the
+# output. They are ignored by default.
+#show_authors = False
+
+# The name of the Pygments (syntax highlighting) style to use.
+pygments_style = 'sphinx'
+
+# A list of ignored prefixes for module index sorting.
+#modindex_common_prefix = []
+
+# If true, keep warnings as "system message" paragraphs in the built documents.
+#keep_warnings = False
+
+# If true, `todo` and `todoList` produce output, else they produce nothing.
+todo_include_todos = False
+
+
+# -- Options for HTML output ----------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+html_theme = 'alabaster'
+
+# Theme options are theme-specific and customize the look and feel of a theme
+# further.  For a list of options available for each theme, see the
+# documentation.
+#html_theme_options = {}
+
+# Add any paths that contain custom themes here, relative to this directory.
+#html_theme_path = []
+
+# The name for this set of Sphinx documents.  If None, it defaults to
+# "<project> v<release> documentation".
+#html_title = None
+
+# A shorter title for the navigation bar.  Default is the same as html_title.
+#html_short_title = None
+
+# The name of an image file (relative to this directory) to place at the top
+# of the sidebar.
+#html_logo = None
+
+# The name of an image file (relative to this directory) to use as a favicon of
+# the docs.  This file should be a Windows icon file (.ico) being 16x16 or 32x32
+# pixels large.
+#html_favicon = None
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['_static']
+
+# Add any extra paths that contain custom files (such as robots.txt or
+# .htaccess) here, relative to this directory. These files are copied
+# directly to the root of the documentation.
+#html_extra_path = []
+
+# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
+# using the given strftime format.
+#html_last_updated_fmt = '%b %d, %Y'
+
+# If true, SmartyPants will be used to convert quotes and dashes to
+# typographically correct entities.
+#html_use_smartypants = True
+
+# Custom sidebar templates, maps document names to template names.
+#html_sidebars = {}
+
+# Additional templates that should be rendered to pages, maps page names to
+# template names.
+#html_additional_pages = {}
+
+# If false, no module index is generated.
+#html_domain_indices = True
+
+# If false, no index is generated.
+#html_use_index = True
+
+# If true, the index is split into individual pages for each letter.
+#html_split_index = False
+
+# If true, links to the reST sources are added to the pages.
+#html_show_sourcelink = True
+
+# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
+#html_show_sphinx = True
+
+# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
+#html_show_copyright = True
+
+# If true, an OpenSearch description file will be output, and all pages will
+# contain a <link> tag referring to it.  The value of this option must be the
+# base URL from which the finished HTML is served.
+#html_use_opensearch = ''
+
+# This is the file name suffix for HTML files (e.g. ".xhtml").
+#html_file_suffix = None
+
+# Language to be used for generating the HTML full-text search index.
+# Sphinx supports the following languages:
+#   'da', 'de', 'en', 'es', 'fi', 'fr', 'h', 'it', 'ja'
+#   'nl', 'no', 'pt', 'ro', 'r', 'sv', 'tr'
+#html_search_language = 'en'
+
+# A dictionary with options for the search language support, empty by default.
+# Now only 'ja' uses this config value
+#html_search_options = {'type': 'default'}
+
+# The name of a javascript file (relative to the configuration directory) that
+# implements a search results scorer. If empty, the default will be used.
+#html_search_scorer = 'scorer.js'
+
+# Output file base name for HTML help builder.
+htmlhelp_basename = 'YAMLTestsinSIESTAdoc'
+
+# -- Options for LaTeX output ---------------------------------------------
+
+latex_elements = {
+# The paper size ('letterpaper' or 'a4paper').
+#'papersize': 'letterpaper',
+
+# The font size ('10pt', '11pt' or '12pt').
+#'pointsize': '10pt',
+
+# Additional stuff for the LaTeX preamble.
+#'preamble': '',
+
+# Latex figure (float) alignment
+#'figure_align': 'htbp',
+}
+
+# Grouping the document tree into LaTeX files. List of tuples
+# (source start file, target name, title,
+#  author, documentclass [howto, manual, or own class]).
+latex_documents = [
+    (master_doc, 'YAMLTestsinSIESTA.tex', 'YAML Tests in SIESTA Documentation',
+     'Yann Pouillon', 'manual'),
+]
+
+# The name of an image file (relative to this directory) to place at the top of
+# the title page.
+#latex_logo = None
+
+# For "manual" documents, if this is true, then toplevel headings are parts,
+# not chapters.
+#latex_use_parts = False
+
+# If true, show page references after internal links.
+#latex_show_pagerefs = False
+
+# If true, show URL addresses after external links.
+#latex_show_urls = False
+
+# Documents to append as an appendix to all manuals.
+#latex_appendices = []
+
+# If false, no module index is generated.
+#latex_domain_indices = True
+
+
+# -- Options for manual page output ---------------------------------------
+
+# One entry per manual page. List of tuples
+# (source start file, name, description, authors, manual section).
+man_pages = [
+    (master_doc, 'yamltestsinsiesta', 'YAML Tests in SIESTA Documentation',
+     [author], 1)
+]
+
+# If true, show URL addresses after external links.
+#man_show_urls = False
+
+
+# -- Options for Texinfo output -------------------------------------------
+
+# Grouping the document tree into Texinfo files. List of tuples
+# (source start file, target name, title, author,
+#  dir menu entry, description, category)
+texinfo_documents = [
+    (master_doc, 'YAMLTestsinSIESTA', 'YAML Tests in SIESTA Documentation',
+     author, 'YAMLTestsinSIESTA', 'One line description of project.',
+     'Miscellaneous'),
+]
+
+# Documents to append as an appendix to all manuals.
+#texinfo_appendices = []
+
+# If false, no module index is generated.
+#texinfo_domain_indices = True
+
+# How to display URL addresses: 'footnote', 'no', or 'inline'.
+#texinfo_show_urls = 'footnote'
+
+# If true, do not generate a @detailmenu in the "Top" node's menu.
+#texinfo_no_detailmenu = False
+
+# -- Further options ------------------------------------------------------
+smartquotes = False

=== added file 'Docs/yaml_tests/implementation.rst'
--- Docs/yaml_tests/implementation.rst	1970-01-01 00:00:00 +0000
+++ Docs/yaml_tests/implementation.rst	2018-05-30 08:59:05 +0000
@@ -0,0 +1,65 @@
+==========================================
+Implementation of the YAML variable dumper
+==========================================
+
+Changes in SIESTA
+-----------------
+
+The YAML variable dumper has been implemented in as an orthogonal way as
+possible from the rest of SIESTA. It consists in:
+
+- a Fortran module producing valid YAML 1.2, codename *m_io_yaml*;
+- the inclusion of the corresponding object file into the MAkefile of SIESTA;
+- a call to the main routine of the module, *siesta_write_yaml* at the end of
+  *siesta.F*.
+
+The call to the I/O routine is both compatible with serial and parallel runs
+of SIESTA:
+
+.. code::
+
+         if (ionode) call siesta_write_yaml()
+
+This strategy is expected to be harmless to any other ongoing developments
+within SIESTA and allows to add the YAML output capability into already
+published versions of SIESTA. Patching such versions will indeed provide
+useful information on possible issues that were previously undetected.
+
+.. note::
+
+   Contrary to XML which has tags and attributes, YAML is just plain text
+   using indenting (think of Python) to define data structures. As a
+   consequence, emitting YAML does **not** introduce any additional dependency
+   of SIESTA on another external library and is much easier to read by humans.
+
+
+Source code
+-----------
+
+The complete source code of the YAML dumper can be found in
+*~siesta/Src/m_io_yaml.F90*. The file is self-documented using the Doxygen
+format.
+
+.. note::
+
+   The Doxygen infrastructure is not merged yet...
+
+
+Documentation
+-------------
+
+To generate the Doxygen documentation, just run *doxygen* from the top
+source directory of SIESTA. The resulting files can then be found within the
+*~siesta/Docs/developers/* directory.
+
+To access the HTML website, look for *index.html* inside the *source/*
+subdirectory. To build the PDF manual, look at the *refman.tex* main document
+inside the *latex/* subdirectory.
+
+.. tip::
+
+   The provided *Doxyfile* has been tested with Doxygen 1.8.11. If you have an
+   earlier version and experience crashes, just add the problematic files to
+   the *EXCLUDE* variable within the *Doxyfile* and re-run doxygen. Repeat
+   this procedure as many times as necessary.
+

=== added file 'Docs/yaml_tests/index.rst'
--- Docs/yaml_tests/index.rst	1970-01-01 00:00:00 +0000
+++ Docs/yaml_tests/index.rst	2018-05-30 08:59:05 +0000
@@ -0,0 +1,22 @@
+.. YAML Tests in SIESTA documentation master file, created by
+   sphinx-quickstart on Thu Jan 18 15:13:04 2018.
+   You can adapt this file completely to your liking, but it should at least
+   contain the root `toctree` directive.
+
+YAML Tests in SIESTA
+====================
+
+.. toctree::
+   :maxdepth: 2
+
+   implementation.rst
+   analysis.rst
+
+
+Indices and tables
+==================
+
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`
+

=== modified file 'Src/Makefile'
--- Src/Makefile	2018-05-15 08:57:29 +0000
+++ Src/Makefile	2018-05-30 08:59:05 +0000
@@ -144,7 +144,7 @@
         setup_ordern_indexes.o\
         m_iodm.o m_iodm_old.o \
 	m_filter.o \
-        m_io.o \
+        m_io.o m_io_yaml.o \
 	m_timer.o \
 	extrae_module.o extrae_eventllist.o\
 	moreParallelSubs.o \
@@ -839,6 +839,7 @@
 m_io.o: sys.o
 m_io_s.o: class_OrbitalDistribution.o class_SpData1D.o class_SpData2D.o
 m_io_s.o: class_Sparsity.o parallel.o precision.o
+m_io_yaml.o: m_energies.o precision.o version.o
 m_iodm.o: class_OrbitalDistribution.o class_SpData2D.o class_Sparsity.o
 m_iodm.o: m_io_s.o m_os.o parallel.o
 m_iodm_old.o: alloc.o files.o m_matio.o m_mpi_utils.o parallel.o parallelsubs.o
@@ -1219,9 +1220,9 @@
 setup_ordern_indexes.o: alloc.o domain_decom.o parallel.o spatial.o
 shaper.o: atmfuncs.o mneighb.o precision.o
 show_distribution.o: atomlist.o parallel.o parallelsubs.o siesta_geom.o sys.o
-siesta.o: alloc.o extrae_module.o m_mpi_utils.o m_steps.o parallel.o
-siesta.o: siesta_analysis.o siesta_end.o siesta_forces.o siesta_init.o
-siesta.o: siesta_move.o
+siesta.o: alloc.o extrae_module.o m_io_yaml.o m_mpi_utils.o m_steps.o
+siesta.o: parallel.o siesta_analysis.o siesta_end.o siesta_forces.o
+siesta.o: siesta_init.o siesta_move.o
 siesta2wannier90.o: files.o m_digest_nnkp.o m_spin.o m_trialorbitalclass.o
 siesta2wannier90.o: parallel.o precision.o siesta_options.o sys.o
 siesta_analysis.o: alloc.o atomlist.o bands.o basis_enthalpy.o dhscf.o files.o

=== added file 'Src/m_io_yaml.F90'
--- Src/m_io_yaml.F90	1970-01-01 00:00:00 +0000
+++ Src/m_io_yaml.F90	2018-05-30 08:59:05 +0000
@@ -0,0 +1,215 @@
+!> \brief YAML output of SIESTA variables for external post-processors.
+!!
+!! This module provides facilities to output SIESTA variables into a YAML
+!! file at the end of a run.
+!!
+!! Variables are dumped in YAML format to provide structured data that
+!! can be read partially or fully, in any order, at the option of the user.
+!!
+!! The resulting output can be processed in C/C++ using LibYAML (found at
+!! https://github.com/yaml/libyaml), in Python through the ruamel.yaml
+!! package (found at https://pypi.python.org/pypi/ruamel.yaml), and in Perl
+!! thanks to the YAML::XS module (found at
+!! http://search.cpan.org/~tinita/YAML-LibYAML-0.69/lib/YAML/XS.pod).
+!!
+!! \author Yann Pouillon
+!! \date 2017-2018
+!! \copyright GNU General Public License version 3
+!!
+!! \note The interest of using the YAML format is that output can be achieved
+!!       without introducing new external dependencies.
+!!
+module m_io_yaml
+
+  use precision, only: dp
+
+  implicit none
+
+  private
+
+  ! Ensure compliance with the YAML 1.2 file format
+  character(len=*), parameter :: CH10 = achar(10)   !< New line
+  character(len=*), parameter :: CH34 = achar(34)   !< Double quote
+  character(len=*), parameter :: YAML_HEADER = "%YAML 1.2"//CH10//"---"//CH10
+  character(len=*), parameter :: YAML_FOOTER = CH10//"..."
+
+  public :: siesta_write_yaml
+
+contains
+
+  !> \brief Creates a YAML file containing SIESTA build parameters and
+  !!        final energy values.
+  !!
+  !! This routine calls io_assign() to get a free unit number and outputs
+  !! valid YAML data structures into the associated OUTVARS.yml file. The
+  !! data consists in two dictionaries:
+  !!   - siesta: build parameters and status of optional features;
+  !!   - energies: full decomposition of the total energy, using exactly
+  !!     the same naming conventions as in the \ref m_energies module.
+  !!
+  !! The resulting file can easily be parsed using the ruamel.yaml Python
+  !! module.
+  !!
+  !! \todo Output forces and stress tensor.
+  subroutine siesta_write_yaml()
+
+    use m_energies
+    use version_info
+
+    implicit none
+
+    logical :: trigger
+    integer :: ierr, yaml_fd
+
+    ! Open YAML document
+    call io_assign(yaml_fd)
+    open(unit=yaml_fd, file='OUTVARS.yml', status='new', action='write', &
+&     access='sequential', form='formatted', iostat=ierr)
+    ! FIXME: Find out why the system sometines reports a failure while things
+    !        have gone perfectly well.
+    !if ( ierr .ne. 0 ) call die('could not open OUTVARS.yml')
+    write(unit=yaml_fd, fmt='(A)') YAML_HEADER
+
+    ! Dump SIESTA information
+    write(unit=yaml_fd, fmt='(A)') "siesta:"
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,3(A))') "version", &
+&     CH34, trim(adjustl(version_str)), CH34
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,3(A))') "arch", &
+&     CH34, trim(adjustl(siesta_arch)), CH34
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,3(A))') "compiler", &
+&     CH34, trim(adjustl(compiler_version)), CH34
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,3(A))') "fflags", &
+&     CH34, trim(adjustl(fflags)), CH34
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,3(A))') "fppflags", &
+&     CH34, trim(adjustl(fppflags)), CH34
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,3(A))') "libs", &
+&     CH34, trim(adjustl(libs)), CH34
+#ifdef MPI
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,3(A))') "build", &
+&     CH34, "mpi", CH34
+#else
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,3(A))') "build", &
+&     CH34, "serial", CH34
+#endif
+#ifdef _OPENMP
+    trigger = .true.
+#else
+    trigger = .false.
+#endif
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,3(A))') "openmp", &
+&     CH34, trim(yesno(trigger)), CH34
+#ifdef USE_GEMM3M
+    trigger = .true.
+#else
+    trigger = .false.
+#endif
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,3(A))') "gemm3m", &
+&     CH34, trim(yesno(trigger)), CH34
+#ifdef CDF
+    trigger = .true.
+#else
+    trigger = .false.
+#endif
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,3(A))') "netcdf", &
+&     CH34, trim(yesno(trigger)), CH34
+#ifdef NCDF_4
+    trigger = .true.
+#else
+    trigger = .false.
+#endif
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,3(A))') "netcdf4", &
+&     CH34, trim(yesno(trigger)), CH34
+#ifdef NCDF_PARALLEL
+    trigger = .true.
+#else
+    trigger = .false.
+#endif
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,3(A))') "netcdf4_mpi", &
+&     CH34, trim(yesno(trigger)), CH34
+#if defined(ON_DOMAIN_DECOMP) || defined(SIESTA__METIS)
+    trigger = .true.
+#else
+    trigger = .false.
+#endif
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,3(A))') "metis", &
+&     CH34, trim(yesno(trigger)), CH34
+#ifdef SIESTA__CHESS
+    trigger = .true.
+#else
+    trigger = .false.
+#endif
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,3(A))') "chess", &
+&     CH34, trim(yesno(trigger)), CH34
+#ifdef SIESTA__ELPA
+    trigger = .true.
+#else
+    trigger = .false.
+#endif
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,3(A))') "elpa", &
+&     CH34, trim(yesno(trigger)), CH34
+#ifdef SIESTA__FLOOK
+    trigger = .true.
+#else
+    trigger = .false.
+#endif
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,3(A))') "flook", &
+&     CH34, trim(yesno(trigger)), CH34
+#ifdef SIESTA__PEXSI
+    trigger = .true.
+#else
+    trigger = .false.
+#endif
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,3(A))') "pexsi", &
+&     CH34, trim(yesno(trigger)), CH34
+
+    ! Dump energies
+    write(unit=yaml_fd, fmt='(A,A)') CH10, "energies:"
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,E24.8)') "Ebs", Ebs
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,E24.8)') "Eions", Eions
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,E24.8)') "Ena", Ena
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,E24.8)') "Ekin", Ekin
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,E24.8)') "Enl", Enl
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,E24.8)') "Eso", Eso
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,E24.8)') "Eldau", Eldau
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,E24.8)') "DEna", DEna
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,E24.8)') "DUscf", DUscf
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,E24.8)') "DUext", DUext
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,E24.8)') "Enegf", DE_NEGF
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,E24.8)') "Exc", Exc
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,E24.8)') "Ecorrec", Ecorrec
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,E24.8)') "Emadel", Emad
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,E24.8)') "Emeta", Emeta
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,E24.8)') "Emolmec", Emm
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,E24.8)') "Ekinion", Ekinion
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,E24.8)') "Eharris", Eharrs+Ekinion
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,E24.8)') "Etot", Etot+Ekinion
+    write(unit=yaml_fd, fmt='(2X,A,":",1X,E24.8)') "FreeEng", FreeE+Ekinion
+
+    ! Close YAML document
+    write(unit=yaml_fd, fmt='(A)') YAML_FOOTER
+    call io_close(yaml_fd)
+
+  end subroutine siesta_write_yaml
+
+  !> \brief Internal function to translate booleans into "yes"/"no" strings.
+  !!
+  !! This function takes a boolean condition as input and returns a string
+  !! corresponding to the boolean value.
+  !!
+  !! \param[in] cond: boolean condition
+  !! \return string equal to "yes" for .true., and "no" for .false.
+  function yesno(cond) result(word)
+
+    logical, intent(in) :: cond
+
+    character(len=3) :: word
+
+    if ( cond ) then
+      word = "yes"
+    else
+      word = "no "
+    end if
+
+  end function yesno
+
+end module m_io_yaml

=== modified file 'Src/siesta.F'
--- Src/siesta.F	2016-08-10 11:29:54 +0000
+++ Src/siesta.F	2018-05-30 08:59:05 +0000
@@ -25,6 +25,7 @@
 #ifdef TRACING_SOLVEONLY
       use extrae_module
 #endif
+      use m_io_yaml, only: siesta_write_yaml
 
       implicit none
 
@@ -91,6 +92,8 @@
       if (ionode) call memory_snapshot("after siesta_analysis")
 #endif
 
+      if (ionode) call siesta_write_yaml()
+
       call siesta_end()
 #ifdef SIESTA__PEXSI
       if (ionode) call memory_snapshot("at end of program")

=== added file 'Tests/Scripts/yaml_compare.py'
--- Tests/Scripts/yaml_compare.py	1970-01-01 00:00:00 +0000
+++ Tests/Scripts/yaml_compare.py	2018-05-30 08:59:05 +0000
@@ -0,0 +1,303 @@
+#!/usr/bin/env python3
+
+from __future__ import print_function
+
+import argparse
+import os
+import sys
+
+from time import gmtime, strftime
+from ruamel.yaml import YAML
+try:
+    from termcolor import colored
+except ImportError:
+    sys.stderr.write("Warning: The termcolor package is unavailable\n")
+    sys.stderr.write("         The --fancy option will be ignored\n")
+    def colored(text, fgcolor, bgcolor=None, **kwargs):
+        return text
+
+
+                    # ------------------------------------ #
+
+
+# Color attributes for colorized reports
+fancy_color = {
+    "????": ["grey", "on_yellow", "bold,dark"],
+    "enfl": ["grey", "on_yellow", "bold,dark"],
+    "eout": ["yellow", "on_red", "bold"],
+    "eref": ["white", "on_red", "bold"],
+    "etol": ["yellow", None, "bold"],
+    "xtfl": ["yellow", "on_green", "bold"],
+    "pass": ["green", None, "bold"],
+    "skip": ["grey", None, "bold"],
+}
+
+# Test status formatter for both BW and color reports
+def fancy_status(stat_str, colorize=False):
+
+    # Maximum status length is 4
+    if ( len(stat_str) >= 4 ):
+        fancy_str = stat_str[0:4]
+    else:
+        fancy_str = stat_str
+
+    # Minimum status length is 4
+    if ( len(fancy_str) == 0 ):
+        fancy_str = "????"
+    if ( len(fancy_str) <= 3 ):
+        fancy_str += " "
+    if ( len(fancy_str) <= 3 ):
+        fancy_str = " " + fancy_str
+    if ( len(fancy_str) <= 3 ):
+        fancy_str += " "
+
+    # Colorize output when requested
+    if ( colorize ):
+        if ( fancy_str in fancy_color ):
+            fgcolor, bgcolor, deco = fancy_color[fancy_str]
+            if ( len(deco) > 0 ):
+                deco = deco.split(",")
+            else:
+                deco = []
+        else:
+            fgcolor = "yellow"
+            bgcolor = "on_blue",
+            deco = ["bold"]
+        if ( bgcolor ):
+            fancy_str = colored(fancy_str.upper(), fgcolor, bgcolor, attrs=deco)
+        else:
+            fancy_str = colored(fancy_str.upper(), fgcolor, attrs=["bold"])
+        fancy_str = colored("[", "white") + fancy_str + colored("]", "white")
+    else:
+        fancy_str = "[" + fancy_str.upper() + "]"
+
+    return fancy_str
+
+
+                    # ------------------------------------ #
+
+
+# Process command-line arguments
+parser = argparse.ArgumentParser(
+    description="Checks SIESTA test results against references")
+parser.add_argument("-a", "--abs-tol", type=float, default=1.0e-7, 
+    help="Absolute tolerance for comparisons (overrides configuration)")
+parser.add_argument("-c", "--config", default="siesta-testsuite.yml",
+    help="Config file containing the SIESTA test suite specifications")
+parser.add_argument("-d", "--details", action="store_true", default=False,
+    help="Display detailed information about each test")
+parser.add_argument("-f", "--fancy", action="store_true", default=False,
+    help="Print fancy colorized report (for terminals supporting it)")
+parser.add_argument("-o", "--output", default="tests-report.yml",
+    help="File to store a YAML version of the report with full details")
+parser.add_argument("-r", "--refdir", default="YAML_Refs",
+    help="Directory containing reference YAML files")
+parser.add_argument("-t", "--testdir", default=".",
+    help="Directory containing the test cases with their outputs")
+parser.add_argument("-y", "--yaml", default=os.path.join("work", "OUTVARS.yml"),
+    help="Relative path where each test stores its YAML data")
+args = parser.parse_args()
+
+# Check command-line arguments
+if ( not os.path.exists(args.config) ):
+    parser.error("config file not found: '%s'" % args.config)
+if ( not os.path.isdir(args.refdir) ):
+    parser.error("reference directory not found: '%s'" % args.refdir)
+if ( not os.path.isdir(args.testdir) ):
+    parser.error("tests directory not found: '%s'" % args.testdir)
+
+# Banner
+print("""\
+========================================
+Summary of the SIESTA Test Suite results
+========================================
+
+Introduction
+------------
+
+Each relevant test case of the SIESTA Test Suite has been run and its results
+compared to a reference file.
+
+In the following, the result of each comparison is represented in a compact
+way at the beginning of each line through a 4-letter keyword surrounded by
+square brackets. It is followed by a description of the correpsonding test,
+condensed by default, verbose if requested.
+
+Here is the meaning of each keyword:
+
+- ENFL: The test case succeeded while it was expected to fail.
+- EOUT: Error reading YAML output file of the test case.
+- EREF: Error reading YAML reference file of the test case.
+- ETOL: Some values are beyond permitted tolerances.
+- PASS: The test case meets all requirements.
+- SKIP: The test case has not been executed.
+- XTFL: The test failed as expected.
+
+A test is successful only if its result is "PASS".
+
+A report with full details of the tests is stored in the output file listed
+below in YAML format.
+
+
+Test suite parameters
+---------------------
+
+The test comparison script has been run with the following parameters:
+
+- Workdir       : %s
+- Configuration : %s
+- Reference dir : %s
+- Test dir      : %s
+- Output file   : %s
+
+
+Results
+-------
+""" % (os.getcwd(), args.config, args.refdir, args.testdir, args.output))
+
+# Load global configuration
+yaml_doc = YAML()
+yaml_cfg = {"tests": []}
+with open(args.config, "r") as cfg_file:
+    yaml_cfg = yaml_doc.load(cfg_file)
+
+# Perform test comparisons
+siesta_tests = []
+siesta_index = {}
+siesta_xfail = []
+for tcase in yaml_cfg["tests"]:
+
+    # Init
+    siesta_index[tcase["name"]] = len(siesta_tests)
+    siesta_tests.append({"name": tcase["name"], "title": tcase["title"]})
+    if ( ("fail_expect" in tcase) and (tcase["fail_expect"] == "yes") ):
+        siesta_xfail.append(tcase["name"])
+
+    # Check that test reference exists
+    ref_path = os.path.join(args.refdir, "%s.yml" % tcase["name"])
+    if ( not os.path.exists(ref_path) ):
+        siesta_tests[-1]["result"] = "eref"
+        siesta_tests[-1]["message"] = "missing reference file"
+        continue
+
+    # Load test reference
+    yaml_doc = YAML()
+    with open(ref_path, "r") as ref_file:
+        yaml_ref = yaml_doc.load(ref_file)
+
+    # Check that test output exists
+    out_path = os.path.join(args.testdir, tcase["name"], args.yaml)
+    if ( not os.path.exists(out_path) ):
+        siesta_tests[-1]["result"] = "eout"
+        siesta_tests[-1]["message"] = "missing output file"
+        continue
+
+    # Load test output
+    yaml_doc = YAML()
+    with open(out_path, "r") as out_file:
+        yaml_out = yaml_doc.load(out_file)
+
+    # Compare energies
+    siesta_tests[-1]["tolerances"] = {}
+    ref_vars = yaml_ref["energies"]
+    out_vars = yaml_out["energies"]
+    tc_good = []
+    tc_fail = []
+    tc_skip = []
+    for (key, val) in ref_vars.items():
+        if ( ("tolerances" in tcase.keys()) and (key in tcase["tolerances"]) ):
+            tc_tol = tcase["tolerances"][key]
+        elif ( key in yaml_cfg["tolerances"] ):
+            tc_tol = yaml_cfg["tolerances"][key]
+        else:
+            tc_tol = args.abs_tol
+        siesta_tests[-1]["tolerances"][key] = tc_tol
+
+        if ( (key in out_vars) and (abs(out_vars[key] - val) < tc_tol) ):
+            tc_good.append(key)
+        else:
+            tc_fail.append(
+              {"name": key, "value": out_vars[key], "expected": val})
+
+    # Store comparison results
+    tc_skip = sorted([item for item in out_vars.keys() if not item in ref_vars.keys()])
+    siesta_tests[-1]["good"] = tc_good
+    siesta_tests[-1]["fail"] = tc_fail
+    siesta_tests[-1]["skip"] = tc_skip
+
+    # Store final test result
+    if ( len(tc_fail) == 0 ):
+        siesta_tests[-1]["result"] = "pass"
+        siesta_tests[-1]["message"] = "all values within tolerance"
+    else:
+        siesta_tests[-1]["result"] = "etol"
+        siesta_tests[-1]["message"] = "some values beyond tolerance"
+
+# Write down full report
+yaml_doc = YAML()
+yaml_doc.default_flow_style=False
+yaml_doc.indent(mapping=4, sequence=4, offset=2)
+report = {"siesta_tests": siesta_tests,
+          "defaults": {"tolerance": args.abs_tol}}
+with open(args.output, "w") as rep_file:
+    rep_file.write("%YAML 1.2\n---\n\n")
+    yaml_doc.dump(report, rep_file)
+    rep_file.write("\n...\n")
+
+# Display test results
+for tcase in siesta_tests:
+
+    if ( tcase["name"] in siesta_xfail ):
+        if ( tcase["result"] == "pass" ):
+            tcase["result"] = "enfl"
+        else:
+            tcase["result"] = "xtfl"
+        tcase["message"] += " (expected to fail)"
+
+    if ( not args.details ):
+        titlen = 65 - len(tcase["name"])
+        if ( len(tcase["title"]) > titlen ):
+            tcase["title"] = tcase["title"][:titlen] + "..."
+
+    if ( args.fancy ):
+        tcase["name"] = colored(tcase["name"], "white", attrs=["bold"])
+        tcase["title"] = colored(tcase["title"], "white", attrs=["dark"])
+
+    print("-", fancy_status(tcase["result"], args.fancy), tcase["name"],
+        "> %s" % tcase["title"])
+
+    if ( args.details ):
+        if ( not tcase["result"] in ["eout", "eref"] ):
+            print("")
+            print("  - Energies/good: %s", tcase["good"])
+            print("  - Energies/fail: %s", tcase["fail"])
+            print("  - Energies/skip: %s", tcase["skip"])
+            print("")
+
+# Display statistics
+ntests = len(siesta_tests)
+ntpass = len([tcase for tcase in siesta_tests if tcase["result"] == "pass"])
+print("""\
+
+
+Statistics
+----------
+
+Global results of the test suite:
+
+- number of tests  : %4d
+- successful tests : %4d (%5.1f%%)
+- failed tests     : %4d (%5.1f%%)
+
+""" % (ntests, ntpass, ntpass*100.0/ntests, ntests-ntpass, (ntests-ntpass)*100.0/ntests))
+
+# Display footer
+print("""\
+.. note::
+
+   This report is valid ReStructuredText, unless you ask for colorized output.
+   You can render it in HTML, PDF and other formats using the Docutils package.
+   Please consult `http://docutils.sourceforge.net/`_ for details.
+
+Document generated on %s (UTC).
+""" % strftime("%Y/%m/%d %H:%M:%S +0000",gmtime()))

=== added directory 'Tests/YAML_Refs'
=== added file 'Tests/YAML_Refs/benzene.yml'
--- Tests/YAML_Refs/benzene.yml	1970-01-01 00:00:00 +0000
+++ Tests/YAML_Refs/benzene.yml	2018-05-30 08:59:05 +0000
@@ -0,0 +1,45 @@
+%YAML 1.2
+---
+
+siesta:
+  version: "siesta-4.1--920"
+  arch: "gfortran-macosx64-openmpi"
+  compiler: "GNU Fortran (Homebrew GCC 7.2.0) 7.2.0"
+  fflags: "mpif90 -O2 -fbacktrace -g"
+  fppflags: "-DCDF -DMPI  -DMPI_TIMING -DF2003"
+  libs: "-L/usr/local/lib -lnetcdff -L/opt/scalapack/openmpi-2.1.2--gfortran-7.2.0/liib -lscalapack -lveclibfort"
+  build: "mpi"
+  openmp: "no"
+  transiesta: "no"
+  gemm3m: "no"
+  netcdf: "yes"
+  netcdf4: "no"
+  netcdf4_mpi: "no"
+  metis: "no"
+  chess: "no"
+  elpa: "no"
+  flook: "no"
+  pexsi: "no"
+
+energies:
+  Ebs:          -0.26343244E+02
+  Eions:           0.11862249E+03
+  Ena:           0.25019919E+02
+  Ekin:           0.53830442E+02
+  Enl:          -0.87719378E+01
+  Eso:           0.00000000E+00
+  Eldau:           0.00000000E+00
+  DEna:          -0.24939449E+01
+  DUscf:           0.27876148E+00
+  DUext:           0.00000000E+00
+  Exc:          -0.24492164E+02
+  Ecorrec:           0.00000000E+00
+  Emadel:           0.00000000E+00
+  Emeta:           0.00000000E+00
+  Emolmec:           0.00000000E+00
+  Ekinion:           0.00000000E+00
+  Eharris:          -0.75251411E+02
+  Etot:          -0.75251411E+02
+  FreeEng:          -0.75251411E+02
+
+...

=== added file 'Tests/YAML_Refs/h2o.yml'
--- Tests/YAML_Refs/h2o.yml	1970-01-01 00:00:00 +0000
+++ Tests/YAML_Refs/h2o.yml	2018-05-30 08:59:05 +0000
@@ -0,0 +1,45 @@
+%YAML 1.2
+---
+
+siesta:
+  version: "siesta-4.1--920"
+  arch: "gfortran-macosx64-openmpi"
+  compiler: "GNU Fortran (Homebrew GCC 7.2.0) 7.2.0"
+  fflags: "mpif90 -O2 -fbacktrace -g"
+  fppflags: "-DCDF -DMPI  -DMPI_TIMING -DF2003"
+  libs: "-L/usr/local/lib -lnetcdff -L/opt/scalapack/openmpi-2.1.2--gfortran-7.2.0/liib -lscalapack -lveclibfort"
+  build: "mpi"
+  openmp: "no"
+  transiesta: "no"
+  gemm3m: "no"
+  netcdf: "yes"
+  netcdf4: "no"
+  netcdf4_mpi: "no"
+  metis: "no"
+  chess: "no"
+  elpa: "no"
+  flook: "no"
+  pexsi: "no"
+
+energies:
+  Ebs:          -0.76982627E+01
+  Eions:           0.59963727E+02
+  Ena:           0.12873605E+02
+  Ekin:           0.25782413E+02
+  Enl:          -0.45540719E+01
+  Eso:           0.00000000E+00
+  Eldau:           0.00000000E+00
+  DEna:          -0.13095670E+00
+  DUscf:           0.53468305E-01
+  DUext:           0.00000000E+00
+  Exc:          -0.82990075E+01
+  Ecorrec:           0.00000000E+00
+  Emadel:           0.00000000E+00
+  Emeta:           0.00000000E+00
+  Emolmec:           0.00000000E+00
+  Ekinion:           0.00000000E+00
+  Eharris:          -0.34238277E+02
+  Etot:          -0.34238277E+02
+  FreeEng:          -0.34238277E+02
+
+...

=== added file 'Tests/YAML_Refs/mno_ldau.yml'
--- Tests/YAML_Refs/mno_ldau.yml	1970-01-01 00:00:00 +0000
+++ Tests/YAML_Refs/mno_ldau.yml	2018-05-30 08:59:05 +0000
@@ -0,0 +1,45 @@
+%YAML 1.2
+---
+
+siesta:
+  version: "siesta-4.1--920"
+  arch: "gfortran-macosx64-openmpi"
+  compiler: "GNU Fortran (Homebrew GCC 7.2.0) 7.2.0"
+  fflags: "mpif90 -O2 -fbacktrace -g"
+  fppflags: "-DCDF -DMPI  -DMPI_TIMING -DF2003"
+  libs: "-L/usr/local/lib -lnetcdff -L/opt/scalapack/openmpi-2.1.2--gfortran-7.2.0/liib -lscalapack -lveclibfort"
+  build: "mpi"
+  openmp: "no"
+  transiesta: "no"
+  gemm3m: "no"
+  netcdf: "yes"
+  netcdf4: "no"
+  netcdf4_mpi: "no"
+  metis: "no"
+  chess: "no"
+  elpa: "no"
+  flook: "no"
+  pexsi: "no"
+
+energies:
+  Ebs:          -0.73133469E+01
+  Eions:           0.97990361E+02
+  Ena:           0.14292867E+02
+  Ekin:           0.73606487E+02
+  Enl:          -0.31590480E+02
+  Eso:           0.00000000E+00
+  Eldau:           0.39836046E-01
+  DEna:          -0.15678393E+01
+  DUscf:           0.10598631E+00
+  DUext:           0.00000000E+00
+  Exc:          -0.33376425E+02
+  Ecorrec:           0.00000000E+00
+  Emadel:           0.00000000E+00
+  Emeta:           0.00000000E+00
+  Emolmec:           0.00000000E+00
+  Ekinion:           0.00000000E+00
+  Eharris:          -0.76479929E+02
+  Etot:          -0.76479929E+02
+  FreeEng:          -0.76480129E+02
+
+...

=== added file 'Tests/siesta-testsuite.yml'
--- Tests/siesta-testsuite.yml	1970-01-01 00:00:00 +0000
+++ Tests/siesta-testsuite.yml	2018-05-30 08:59:05 +0000
@@ -0,0 +1,258 @@
+%YAML 1.2
+---
+
+# Default test parameters:
+#
+# - category   : basic = "just run SIESTA", advanced = "complex workflow"
+# - fail_allow : whether it is OK for a test to fail
+# - fail_expect: whether a test is expected to fail (new feature development)
+# - program    : executable program to run
+# - timeout    : maximum run time allowed (in case the program hangs)
+#
+# All these parameters can be overridden individually for each test.
+#
+defaults:
+  category: basic
+  fail_allow: no
+  fail_expect: no
+  program: siesta
+  timeout: 300
+
+# Term-wise tolerances (can be overridden individually for each test)
+# Note: the following values were imported from ~siesta/Tests/tolerances.dat
+tolerances:
+  DEna:   0.015
+  DUscf:  0.001
+  Eharris: 0.003
+  Ekin:   0.017
+  Ena:     0.003
+  Enl:    0.005
+  Etot:   0.003
+  Exc:    0.003
+  FreeEng: 0.005
+  Uscf:   0.015
+
+# Specifications for each test case:
+#
+# - auxdata : auxilliary files needed by the test
+# - name    : short name for the test (should match the test directory name)
+# - title   : one-sentence description of the test
+# - triggers: optional SIESTA features (see the "siesta" section of any
+#             OUTVARS.yml file)
+#
+# The "name" and "title" fields are mandatory. The rest is optional.
+#
+tests:
+  - name: "32_h2o"
+    title: "32 water molecules for simplex minimization"
+  - name: "ag"
+    title: "Untitled"
+  - name: "ar2_vdw"
+    title: "Unconverged optimization of the Argon dimer with vdW-DF"
+  - name: "batio3"
+    title: "Barium titanate cubic structure"
+  - name: "benzene"
+    title: "Benzene molecule"
+  - name: "bessel"
+    title: "Water molecule with Bessel orbitals"
+  - name: "born"
+    title: "Untitled"
+    auxdata: ["FC.fdf"]
+  - name: "born_spin"
+    title: "Untitled"
+    auxdata: ["FC.fdf"]
+  - name: "carbon_nanoscroll"
+    title: "80-5-20 carbon nanoscroll"
+  - name: "ch4"
+    title: "FCC Methane with vdW pre-packaged molecules"
+  - name: "chargeconf-h2o"
+    title: "Water molecule with charge confinement"
+  - name: "constant_volume"
+    title: "MgCO3 SZ in variable primitive cell with constant volume"
+  - name: "dipole_correction"
+    title: "Automatic dipole correction of a layer of water molecules"
+  - name: "fe"
+    title: "BCC iron, non-linear partial-core correction, Pulay mixing"
+  - name: "fe_broyden"
+    title: "BCC iron, non-linear partial-core correction, Broyden mixing"
+  - name: "fe_clust_noncollinear"
+    title: "Linear Fe3 cluster, LDA"
+  - name: "fe_clust_noncollinear-gga"
+    title: "Linear Fe3 cluster, GGA"
+  - name: "fe_cohp"
+    title: "BCC iron, non-linear partial-core correction, COOP curves"
+  - name: "fe_noncol_kp"
+    title: "Linear Fe3 cluster with k-points"
+  - name: "fen"
+    title: "Bulk FeN in ZB structure , COOP curves"
+  - name: "fire_benzene"
+    title: "Benzene molecule with FIRE optimizer"
+  - name: "floating"
+    title: "Water molecule with Floating H orbitals"
+  - name: "force_2"
+    title: "Untitled"
+    auxdata: ["FC.fdf"]
+  - name: "force_constants"
+    title: "Untitled"
+    auxdata: ["FC.fdf"]
+  - name: "gate_G_charge"
+    title: "Untitled"
+  - name: "gate_G_hartree"
+    title: "Untitled"
+  - name: "ge111"
+    title: "Germanium 111 slab with 2x8 reconstruction"
+  - name: "ge_fatbands_so"
+    title: "Bulk Ge with SOC and various types of WFS output"
+    tags: ["WIP"]
+  - name: "graphite_c6"
+    title: "Untitled"
+  - name: "graphite_vdw_df"
+    title: "Unconverged graphite with vdw-DF"
+  - name: "h2_bessel"
+    title: "H2 molecule with a Bessel orbital in the middle"
+  - name: "h2o"
+    title: "Water molecule"
+  - name: "h2o-pop"
+    title: "Untitled"
+  - name: "h2o-rhog-mixing"
+    title: "Untitled"
+  - name: "h2oZ"
+    title: "Untitled"
+  - name: "h2o_2"
+    title: "Water molecule, changing the numbering of atoms"
+  - name: "h2o_4"
+    title: "Cluster of 4 water molecules, compute mu"
+  - name: "h2o_am05"
+    title: "Water molecule optimization with the AM05 functional"
+  - name: "h2o_basis"
+    title: "Water molecule with various basis generation options"
+  - name: "h2o_coop"
+    title: "Water molecule, COOP curves"
+  - name: "h2o_dipole"
+    title: "Dipole correction of a water molecule from DOI:10.1103/PhysRevB.59.12301"
+  - name: "h2o_dipole2"
+    title: "Dipole correction of a water molecule from DOI:10.1103/PhysRevB.59.12301"
+  - name: "h2o_dos"
+    title: "Untitled"
+  - name: "h2o_filteret_basis"
+    title: "Water molecule using a Filteret basis set"
+  - name: "h2o_findp_bug"
+    title: "Pre-findp bug example"
+  - name: "h2o_netcdf"
+    title: "NetCDF DMHS history of a water molecule"
+    triggers: ["netcdf"]
+  - name: "h2o_op_broyden"
+    title: "Broyden optimization of a water molecule"
+  - name: "h2o_orderN"
+    title: "Order-N optimization of a water molecule"
+  - name: "h2o_radialgrid"
+    title: "Example of accurate radial grid use"
+  - name: "h2o_reparam"
+    title: "Pseudopotential reparametrization of a water molecule"
+  - name: "h3po4_1"
+    title: "H3PO4 molecule optimization changing species ordering"
+  - name: "h3po4_2"
+    title: "H3PO4 molecule optimization changing species ordering"
+  - name: "h3po4_3"
+    title: "H3PO4 molecule optimization changing species ordering"
+  - name: "h_chain"
+    title: "Hydrogen chain"
+  - name: "h_chain2"
+    title: "Hydrogen chain with 2 atoms per cell and SZ basis"
+  - name: "md_anneal"
+    title: "Harris-functional annealing of a primitive c-Si supercell"
+    auxdata: ["md_anneal.STRUCT_IN"]
+  - name: "md_nose"
+    title: "Nosé MD of a MgCO3 primitive cell"
+  - name: "md_npr"
+    title: "Nosé-Parinello-Rahman MD of a MgCO3 primitive cell"
+  - name: "md_pr"
+    title: "Parinello-Rahman MD of a MgCO3 primitive cell"
+  - name: "md_verlet"
+    title: "Verlet MD of a MgCO3 primitive cell"
+  - name: "mgc-force"
+    title: "One-shot forces of a MgCO3 primitive cell"
+  - name: "mgco3"
+    title: "Default MD of a MgCO3 primitive cell"
+  - name: "mix_broyden"
+    title: "Linear Fe3 cluster, Broyden mixing"
+  - name: "mix_linear"
+    title: "Linear Fe3 cluster, linear mixing"
+  - name: "mix_pulay"
+    title: "Linear Fe3 cluster, Pulay mixing"
+  - name: "mno_ldau"
+    title: "Ferromagnetic Manganese oxide, LDA+U"
+  - name: "n_chain"
+    title: "Nitrogen atom chain"
+  - name: "nanotube-c-5-0"
+    title: "Metallic (5,0) carbon nanotube"
+  - name: "oxyn"
+    title: "Molecule made up of ON hybrids"
+  - name: "pb_bulk"
+    title: "Bulk lead"
+  - name: "pb_bulk_1000"
+    title: "Bulk lead"
+  - name: "pb_filter_cutoff"
+    title: "Bulk lead, cutoff-filtered"
+  - name: "pb_filter_tol"
+    title: "Bulk lead, tolerance-filtered"
+  - name: "ptcda"
+    title: "Untitled"
+  - name: "ptcda-au"
+    title: "Untitled"
+  - name: "scfmix"
+    title: "Linear Fe3 cluster, various mixing methods"
+  - name: "si-optical"
+    title: "Optical properties of bulk Si"
+  - name: "si001"
+    title: "Si(100), asymmetric 2x1 reconstruction"
+  - name: "si001-diags"
+    title: "Untitled"
+  - name: "si001-ldos"
+    title: "Untitled"
+  - name: "si111-spinpol"
+    title: "Untitled"
+  - name: "si2x1h"
+    title: "Untitled"
+  - name: "si2x1h-dipole"
+    title: "Untitled"
+  - name: "si2x1h-dipole-gcs"
+    title: "Untitled"
+  - name: "si2x1h-quench"
+    title: "Untitled"
+  - name: "si64"
+    title: "Untitled"
+  - name: "si64_coop"
+    title: " Cubic supercell of 64 Si atoms, COOP curves"
+  - name: "si_bandpoints"
+    title: "Bulk Silicon to test BandPoints"
+  - name: "si_coop"
+    title: "Si chain for the calculation of COOP curves"
+  - name: "si_fatbands"
+    title: "Bulk Silicon with various types of WFS output"
+  - name: "si_pdos_kgrid"
+    title: "bulk silicon with extra sampling for PDOS"
+  - name: "sic-slab"
+    title: "Hydrogen/SiC surface"
+  - name: "sih"
+    title: "Hydrogen in a supercell of 64 Silicon atoms"
+  - name: "sih-mrrr"
+    title: "Hydrogen in a supercell of 64 Silicon atoms, MRRR diagonalizer"
+  - name: "sih_fire"
+    title: "Hydrogen in a supercell of 64 Silicon atoms, FIRE quenching"
+  - name: "sih_op_broyden"
+    title: "Hydrogen in a supercell of 64 Silicon atoms, Broyden optimization"
+  - name: "sinw"
+    title: "Si Nanowire grown along the 110 direction"
+  - name: "sinw_2"
+    title: "Untitled"
+  - name: "var_cell"
+    title: "Broyden optimization of MgCO3 in a variable primitive cell"
+  - name: "wannier"
+    title: "Occupied bonding Wannier functions of Silicon"
+    auxdata: ["wannier.nnkp"]
+  - name: "zmatrix"
+    title: "Convoluted example of zmatrix input"
+    auxdata: ["Zmatrix.data"]
+
+...

=== modified file 'version.info'
--- version.info	2018-05-25 11:13:25 +0000
+++ version.info	2018-05-30 08:59:05 +0000
@@ -1,1 +1,4 @@
-siesta-4.1--920
+siesta-4.1--920--yaml-923
+
+
+