ManifestDestiny
===============

Universal manifests for Mozilla test harnesses


What is ManifestDestiny?
------------------------

What ManifestDestiny gives you is:

* manifests are (ordered) lists of tests
* tests may have an arbitrary number of key, value pairs
* the parser returns an ordered list of test data structures, which
  are just dicts with some keys.  For example, a test with no
  user-specified metadata looks like this::

  [{'path':
    '/home/jhammel/mozmill/src/ManifestDestiny/manifestdestiny/tests/testToolbar/testBackForwardButtons.js',
    'name': 'testToolbar/testBackForwardButtons.js', 'here':
    '/home/jhammel/mozmill/src/ManifestDestiny/manifestdestiny/tests',
    'manifest': '/home/jhammel/mozmill/src/ManifestDestiny/manifestdestiny/tests',}]

The keys displayed here (path, name, here, and manifest) are reserved
words for ManifestDestiny and any consuming APIs.


Manifest Format
---------------

Manifests are .ini file with the section names denoting the path
relative to the manifest::

 [foo.js]
 [bar.js]
 [fleem.js]

The sections are read in order. In addition, tests may include
arbitrary key, value metadata to be used by the harness.  You can also
have a ``[DEFAULT]`` section that will give key, value pairs that will
be inherited by each test unless overridden::

 [DEFAULT]
 type = restart

 [lilies.js]
 color = white

 [daffodils.js]
 color = yellow
 type = other
 # override type from DEFAULT

 [roses.js]
 color = red

You can also include other manifests::

 [include:subdir/anothermanifest.ini]


Data
----

Manifest Destiny gives tests as a list of dictionary (in python
terms). 

* path: full path to the test
* name: short name of the test; this is the (usually) relative path
  specified in the section name
* here: the parent directory of the manifest
* manifest: the path to the manifest containing the test

This data corresponds to a one-line manifest::

 [testToolbar/testBackForwardButtons.js]

If additional key, values were specified, they would be in this dict
as well.

Outside of the reserved keys, the remaining key, values
are up to convention to use.  There is a (currently very minimal)
generic integration layer in ManifestDestiny for use of all tests.
For instance, if the 'disabled' key is present, you can get the set of
tests without disabled (various other queries are doable as well).

Since the system is convention-based, the harnesses may do whatever
they want with the data.  They may ignore it completely, they may use
the provided integration layer, or they may provide their own
integration layer.  This should allow whatever sort of logic they
want.  For instance, if in yourtestharness you wanted to run only on
mondays for a certain class of tests::

 tests = []
 for test in manifests.tests:
     if 'runOnDay' in test:
        if calendar.day_name[calendar.weekday(*datetime.datetime.now().timetuple()[:3])].lower() == test['runOnDay'].lower():
            tests.append(test)
     else:
        tests.append(test)

To recap:
* the manifests allow you to specify test data
* the parser gives you this data
* you can use it however you want or process it further as you need

Tests are denoted by sections in an .ini file (see
http://hg.mozilla.org/automation/ManifestDestiny/file/tip/manifestdestiny/tests/mozmill-example.ini). 

Additional manifest files may be included with a [include:] directive::

 [include:path-to-additional-file.manifest]

The path to included files is relative to the current manifest.

The ``[DEFAULT]`` section contains variables that all tests inherit from.

Included files will inherit the top-level variables but may override
in their own ``[DEFAULT]`` section.


Creating Manifests
------------------

ManifestDestiny comes with a console script, ``manifestparser create``, that
may be used to create a seed manifest structure from a directory of
files.  Run ``manifestparser help create`` for usage information.


Copying Manifests
-----------------

To copy tests and manifests from a source::

 manifestparser [options] copy from_manifest to_directory -tag1 -tag2 --key1=value1 key2=value2 ...


Upating Tests
-------------

To update the tests associated with with a manifest from a source
directory::

 manifestparser [options] update manifest from_directory -tag1 -tag2 --key1=value1 --key2=value2 ...


Tests
-----

ManifestDestiny includes a suite of tests:

http://hg.mozilla.org/automation/ManifestDestiny/file/tip/manifestdestiny/tests

``test_manifest.txt`` is a doctest that may be helpful in figuring out
how to use the API.  Tests are run via ``python test.py``.


CLI
---

Run ``manifestparser help`` for usage information.

To create a manifest from a set of directories::

 manifestparser [options] create directory <directory> <...> [create-options]

To output a manifest of tests::

 manifestparser [options] write manifest <manifest> <...> -tag1 -tag2 --key1=value1 --key2=value2 ...

To copy tests and manifests from a source::

 manifestparser [options] copy from_manifest to_manifest -tag1 -tag2 --key1=value1 key2=value2 ...

To update the tests associated with with a manifest from a source
directory::

 manifestparser [options] update manifest from_directory -tag1 -tag2 --key1=value1 --key2=value2 ...
