#

# Saving and Restoring States of CLHEP Random Engines and Distributions

We have added new (as of late 2004) methods to the CLHEP Random classes allowing more convenient and natural saving and restoring of engines and distributions. Some inconvenient features of the original methods are:

- The `saveStatus` and `restoreStatus` methods are based on a named file, so that each distribution saved must be placed in its own file.
- Since opening the file is left to the ``restoreStatus`` method, there can be no user-controlled flexibility in how to deal with trouble (such as file not found) in opening that file.
- Each distribution saves the state of the engine it is using. If several distributions share a single instance of an engine, then this wastes I/O time and space on multiple copies of the same data.
- There was never a way to save an engine, and then have another program instantiate a HepRandomEngine* and read back the state to create an engine of the appropriate type, without knowing what sort of engine was saved. We call this functionallity an "engine factory."
- For cases where the user program has its own scheme to persist numbers (for example, unsigned longs), there is no way to cause an engine to produce its state (or restore from a saved state) as a vector of unsigned longs.

The usual idiom for saving data would be to write to an ostream and read from an istream. It is assumed that users know how to construct an fstream (ofstream or ifstream) to work with a file, and using these fstreams as ostreams and istreams.

We have added methods that stick to this idiom, which greatly helps matters. (In fact, the first two of these inconveniences are immediately resolved, and the structure is right for dealing with the third.)

We have also added engine factory capability to deal with anonymous engine saves, and we have given every engine the ability to save and restore to/from a vector of unsigned longs.

- Use Cases
- Added Methods
- How the Use Cases Can Be Coded

## Use Cases

Each of these motivating use cases are based on the following theme: An experiment wishes, when running a simulation job in "debuggable" mode, to capture before each event enough information that a subsequent job can restart at an arbitrary critical event and get the same results (or more typically, exhibit the same problem for investigation). The states of all random engines evolve across events, so this state information must be saved with each event (and restored for the critical event).

There have been occasional requests for methods to let a job restore a "generic" engine (that is, to start using an engine without knowing at compile time which type of engine will have been saved). Some of these added methods provide that capability in special circumstances. In general, however, we make the assumption that the restoring user knows the classes which have been saved. If the restore does not match the save, this will be indicated in the usual ways; in the Random package, this includes output to `cerr` and setting the state of the restoring istream to "bad."

### One static distribution

The user employs one static distribution, calling (say) `RandGauss::shoot()`. She wishes to save/restore the full state (including underlying engine) of just that distribution. This is completely analogous to doing ``RandGauss::saveState("filename"), but it is desired that this state be added to a file rather than placed in its own file.

### One instance of a distribution

The user has an instance of a distribution, which uses some instance of an engine. She wishes to save/restore its full state, including that of the underlying engine.

### Some distributions sharing various engines

The user has several instances of a distributions of various types, (and/or uses the `shoot()` method of several static distributions) and one or more instances of engines of various types. To conserve I/O space, she wishes to separately save each engine and the non-engine-dependant state of each distribution. Since she knows which engines are used by which distributions, she will be able to restore the full states of all the distributions after restoring individual engine and distribution states.

### All the static distributions

A multi-module program written by a collaboration makes extensive use of the `shoot()` method of various static distributions. They wish to make a single call to save/restore the states of all the static distributions in the CLHEP Randoms package. The program restoring the states of the distributions and the common underlying engine has no a priori knowledge of which type of the underlying engine has been set.

If the static distributions each used a separate engine, this method would need to ensure that states of shared engines would be saved only once, and that all needed engine instances would be saved. However, as currently implemented, every static distribution shares one common engine. Since the (non-engine) state of a distribution itself is miniscule, and only a couple of distributions contain state information other than non-standard distribution parameters (which do not apply to static distributions), the "wasted" cost of saving any unused static distributions is acceptable.

### One engine of unknown type

An application wishes to deal with an engine restored from a saved engine state, without compile-time knowledge of which engine type this will be. For example, the saved engine may be a product of a simulation job where the engine created was determined by run-time input.

### Saving and restoring state via a vector of unsigned longs

An experiment framework has a means of imbedding an array of numbers into the data for an event, and would wish to use this to hold the state of the engine(s) used. Here, the model of saving to a stream is less convenient and much less efficient. HepMC assumes random engines can produce their states in the form of a vector of unsigned longs.

## Added Methods in CLHEP Random

There are two types of save and restore activities on a distribution, just as there are two modes of using a distribution to generate a random variate, depending on whether you want the static distribution or an instance of the distribution. For the generation activity, the methods are Class`::shoot()` to use the static distribution, and instance`.fire()` to use an instance.

For these added methods which deal with saving and restoring to an fstream, the static methods all take the stream as an argument, while the instance methods use the `ostream <<` whatever or `istream >>` whatever syntax.

As in the case of variate generation, all the methods dealing with just an engine are based on an instance.

### Static methods in the distribution classes

Each distribution has six new static methods, which override static methods of the same names in the `HepRandom` class:

    static ostream& saveFullState   (ostream& os) const;
    static istream& restoreFullState(istream& is);

Saves (or restores) the state of the distribution, including the engine used when `shoot()` is invoked and any additional cached data, such as the second gaussian in the Box-Mueller method used by RandGauss.

    static ostream& saveDistState   (ostream& os) const;
    static istream& restoreDistState(istream& is);

Saves (or restores) the static state of the distribution, including cached data used in the course of the `shoot()` method, so that if the engine used is also restored, the full state will have been replicated. For distributions which do not distributions contain cached data (all distributions other than RandGauss, RandFlat, and RandBit) this method has no effect.

    static ostream& saveStaticRandomStates (ostream& os) const;

Determines the identity of `theEngine` used in the static `shoot()` method of the distributions. Saves the state of that engine, followed by the static state of each distribution, such that the states can be restored by `restoreStaticRandomStates`.

    static istream& restoreStaticRandomStates(istream& is);

Restores the cached data for every static distribution and the engine used as `theEngine` in the static `shoot()` method of the distributions.

If, at the point of calling `restoreStaticRandomStates`, the common engine has been set to an engine of the type matching that saved, then that engine's state will be changed to the saved state.

If the two engine types do not match, a new engine of the saved type will be instantiated, and attached to the static distribuitions via a `setTheEngine()` call. The previously existing engine will not be deleted (since it might be shared by some distribution objects), and the new engine is a post-startup new object that won't automatically be deleted, so this may be detected by tools such as Purify as a memory leak.

### Instance methods in the distribution classes

    ostream& operator << (ostream& os, const HepRandom & distribution);
    istream& operator >> (istream& is,       HepRandom & distribution);

Saves (or restores) the state of this instance of the distribution, including cached data used in the course of the `fire()` method, so that if the engine used is also restored, the full state will have been replicated.

    RandomEngine * HepRandom::engine();

Provides a pointer to engine underlying this instance of distribution. This is needed to save the full state of this instance of distribution.

Conventionally, when saving the full state of a distribution, one should follow the order used by the methods for static distributions, which save the engine and then the remainder of the distribution state.

### Instance methods in the engine classes

Some added methods of engine are provided, as tools for implementation of "generic engine" capability:

    string RandomEngine::name();

Provides a string identifying the engine type.

    istream& RandomEngine::getName(istream&);

Reads a string identifying the engine type, and verifies that it is the expected identification.

    istream& RandomEngine::getState(istream&);

Inputs engine state.

    ostream& RandomEngine::put(ostream&);
    istream& RandomEngine::get(istream&);

Outputs or inputs engine state, including identifying name.
Used by operator<< and operator>>.

    vector < unsigned long > RandomEngine::put() const;
    bool RandomEngine::get(const vector<unsigned long> &);

Writes or sets engine state, including 32-bit identifier, to a vector of unsigned longs.

Methods for I/O of engine states to streams are already present. Though not "additional methods," this are listed here because they are logically on the same footing as the distribution I/O methods, and will appear in the section describing how to code the use cases.

    ostream& operator << (ostream& os, const RandomEngine & engine);
    istream& operator >> (istream& is,       RandomEngine & engine);

Saves (or restores) the state of the engine, including an identifying string to help catch mismatches where erroneous code tries to use an engine state as the state of a different kind of engine.

Finally, the base class HepRandomEngine has a pair of methods to obtain a pointer to a new engine based on "generic" or anonymous engine input:

    static HepRandomEngine* HepRandomEngine::newEngine(istream& is);
    static HepRandomEngine* HepRandomEngine::newEngine
    				(const vector<unsigned long> & v);

Based on the contents of the istream or vector, which must contain the output of saving some engine, this instantiates a new engine of the saved type, and returns a pointer to that object. The semantics are that of new: It is the user's responsibility to invoke delete when the engine is no longer needed.
Note that although the engine type is deduced from the input, if the saving program had any non-trivial engine sharing among multiple distribution objects, the restoring program would have to set up the same sharing to get the same subsequent random sequence behaviors.

### Caveat concerning RandEngine

In the case of an engine based on system random generator engine whose internal state is not controllable by the Random package, such as `RandEngine`, the additional save/restore capability is still presented. However, in such a case, the only way to achieve the restore is to start in the original seed state, and fire off however many variates were requested prior to the save. In the case of restoring state based on a late event in a long job, this may be quite inefficient. Therefore, we heavily recommend against using `RandEngine` in applications involving saving and restoring engine states.

## How the Use Cases Can Be Coded

### One static distribution

The "saving job" does:

      DualRand e(24681357);         // Illustrating that the engine need not
      RandGauss::setTheEngine (&e); // be the default engine for RandGauss.
      ofstream fs(filename);        // Open it, or pass an ofstream already in use
      codeUsingRandGaussManyTimes();

      RandGauss::saveFullState(fs);

The "restoring job" does:

      DualRand e();
      RandGauss::setTheEngine (e);
      ifstream file(filename);      // Open it, or pass an ifstream already in use

      RandGauss::restoreFullState(fs);

### One instance of a distribution

The "saving job" does:

      DualRand e(24681357);
      RandGauss g(e);               // Normal distribution, based on engine e
      ofstream fs(filename);
      codeUsing_g_ManyTimes();

      fs << g.engine() << g;

The "restoring job" does:

      DualRand e();
      RandGauss g(e);
      ifstream fs(filename);

      fs >> g.engine() >> g;

### Some distributions sharing various engines

The "saving job" does:

      DualRand e1(24681357);
      MTwistEngine e2(135797531);
      RandGauss g(e1);
      RandFlat  f(e1);
      RandBit   b(e2);
      ofstream fs(filename);
      codeUsing_g_f_and_b();

      fs << e1 << e2 << g << f << b;

The "restoring job" does:

      e = new DualRand();
      RandGauss g(e);
      ifstream fs(filename);

      fs >> e1 >> e2 >> g >> f >> b;

To save and restore successfully in the general case, you must know not only which types of engines underlie the various distributions, but also which distributions share an engine.

### All the static distributions

The "saving job" does:

      e = new DualRand(24681357);   // illustrating that the some distributions
      RandGauss::setTheEngine (e);  // need not use the default engine
      ofstream fs(filename);
      codeUsingVariousDistributions();

      saveStaticRandomStates(fs);

The "restoring job" does:

      e = new DualRand();
      RandGauss::setTheEngine (e);
      ifstream file(filename);

      restoreStaticRandomStates(fs);

### One engine of unknown type

The "saving job" does:

      DualRand *     e1 = new DualRand(24681357);
      RanecuEngine * e2 = new RanecuEngine(24563291);
      HepRandomEngine *eptr;
      if (someControl) eptr = e1; else eptr = e2;
      RandGauss g(*eptr);
      ofstream fs(filename);
      codeUsing_g_ManyTimes();

      fs << g.engine() << g;

      delete e1; delete e2; // (when g goes out of scope)

The "restoring job" does:

      ifstream fs(filename);

      HepRandomEngine * eptr = HepRandomEngine::newEngine(fs);
      if (eptr==0) throw "problem inputting an engine";
      RandGauss g(*eptr);

      codeUsing_g();  // using g without knowing or caring
                      // what type the engine is
      delete eptr;    // (delete engine when g goes out of scope)

### Saving and restoring state via a vector of unsigned longs

The "saving job" does:

      HepJamesRandom e = new HepJamesRandom(12343211);
      codeUsing_e (e);
      std::vector < unsigned long > v;

      v = e.put();

      eventData.add_a_vector (v);

The "restoring job" does:

      eventData.add_a_vector (v);
      HepJamesRandom e;

      bool ok = e.get(v);
      if (!ok) throw "problem restoring an engine";

      codeUsing_e(e);

Author:

Last modified: March 15, 2005