Manager

class glompo.core.manager.GloMPOManager[source]

Provides the main interface to GloMPO. The manager runs the optimization and produces all the output.

The manager is not initialised directly with its settings (__init__() accepts no arguments). Either use setup() to build a new optimization or load_checkpoint() to resume an optimization from a previously saved checkpoint file. Alternatively, class methods new_manager() and load_manager() are also provided. Two equivalent ways to setup a new manager are shown below:

manager = GloMPOManager()
manager.setup(...)

manager = GloMPOManager.new_manager(...)
aggressive_kill

If True and proc_backend is True, child processes are forcibly terminated via SIGTERM. Otherwise, a termination message is sent to the optimizer to shut itself down.

allow_forced_terminations

True if the manager is allowed to force terminate optimizers which appear non-responsive (i.e. do not provide feedback within a specified period of time.

Type:bool
bounds

(Min, max) tuples for each parameter being optimized beyond which optimizers will not explore.

Type:Sequence[Bound]
checkpoint_control

GloMPO object containing all checkpointing settings if this feature is being used.

Type:CheckpointingControl
checkpoint_history

Set of names of checkpoints constructed by the manager.

Type:Set[str]
conv_counter

Count of the number of optimizers which converged according to their own configuration (as opposed to being terminated by the manager).

Type:int
converged

True if the conditions of convergence_checker have been met.

Type:bool
convergence_checker

GloMPO object which evaluates whether conditions are met for overall manager termination.

Type:BaseChecker
cpu_history

History of CPU percentage usage snapshots (taken every status_frequency seconds). This is the CPU percentage used only by the process and its children not the load on the whole system.

Type:List[float]
dt_ends

Records the end of each optimization session for a problem optimized through several checkpoints.

Type:List[datetime.datetime]
dt_starts

Records the start of each optimization session for a problem optimized through several checkpoints.

Type:List[datetime.datetime]
end_timeout

Amount of time the manager will wait to join child processes before forcibly terminating them (if children are processes) or allowing them to eventually crash out themselves (if children are threads). The latter is not recommended as essentially these threads can become orphaned and continue to use resources in the background. Unfortunately, threads cannot be forcibly terminated.

Type:float
f_counter

Number of times the optimization task has been evaluated.

Type:int
hunt_counter

Count of the number of times the manager has evaluated killing_conditions in an attempt to terminate one of its children.

Type:int
hunt_frequency

Frequency (in terms of number of function evaluations) between manager ‘hunts’ (i.e. evaluation of killing_conditions in an attempt to terminate children.

Type:int
hunt_victims

Mapping of manager-killed optimizer ID numbers and timestamps when they were terminated.

Type:Dict[int, float]
incumbent_sharing

If True the manager will send iteration information about the best ever seen solution to all its children whenever this is updated.

Type:bool
is_log_detailed

If True optimizers will attempt to call a task’s detailed_call() method and save the expanded return to the log.

Type:bool
killing_conditions

GloMPO object which evaluates whether an optimizer meets its conditions to be terminated early.

Type:BaseHunter
last_hunt

Evaluation number at which the last hunt was executed.

Type:int
last_iter_checkpoint

f_counter of last attempted checkpoint (regardless of success or failure)

Type:int
last_opt_spawn

Tuple of f_counter and o_counter at which the last child optimizer was started.

Type:Tuple[int, int]
last_status

Timestamp when the last logging status message was printed.

Type:float
last_time_checkpoint

Timestamp of last attempted checkpoint (regardless of success or failure)

Type:float
load_history

History of system load snapshots (taken every status_frequency seconds). This is is a system wide value, not tied to the specific process.

Type:List[Tuple[float, float, float]]
logger

GloMPO has built-in logging to allow tracking during an optimization (see Logging Messages). This attribute accesses the manager logger object.

Type:logging.Logger
max_jobs

Maximum number of calculation ‘slots’ used by all the child optimizers. This generally equates to the number of processing cores available which the child optimizers may fill with threads or processes depending on their configuration. Alternatively, each child optimizer may work serially and take one of these slots.

Type:int
mem_history

History of memory usage snapshots (taken every status_frequency seconds). Details memory used by the process and its children.

Type:List[float]
n_parms

Dimensionality of the optimization problem.

Type:int
o_counter

Number of optimizers started.

Type:int
opt_crashed

True if any child optimizer crashed during its execution.

Type:bool
opt_log

GloMPO object collecting the entire iteration history and metadata of the manager’s children.

Type:BaseLogger
opt_selector

Object which returns an optimizer class and its configuration when requested by the manager. Can be based on previous results delivered by other optimizers.

Type:BaseSelector
optimizer_queue

Common concurrency tool into which all results are paced by child optimizers.

Type:queue.Queue
opts_daemonic

True if manager children are spawned as daemons. Default is True but can be set to False if double process layers are needed (see Parallelism for more details).

Type:bool
overwrite_existing

True if any old files detected in the working directory maybe be deleted when the optimization run begins.

Type:bool
proc_backend

True if the manager children are spawned as processes, False if they are spawned as threads.

Type:bool
result

Incumbent best solution found by any child optimizer.

Type:Result
scope

GloMPO object presenting the optimization graphically in real time.

Type:Optional[GloMPOScope]
spawning_opts

True if the manager is allowed to create new children. The manager will shutdown if all children terminate and this is False.

Type:bool
split_printstreams

True if the printstreams for children are redirected to individual files (see Outputs).

Type:bool
status_frequency

Frequency (in seconds) with which a status message is produced for the logger.

Type:float
summary_files

Logging level indicating how much information is saved to disk.

Type:int
t_end

Timestamp of the ending time of an optimization run.

Type:float
t_start

Timestamp of the starting time of an optimization run.

Type:float
t_used

Total time in seconds used by previous optimization runs. This will be zero unless the manager has been loaded from a checkpoint.

Type:float
task

Function being minimize by the optimizers.

Type:Callable[[Sequence[float]], float]
visualisation

True if the optimization is presented graphically in real time using a GloMPOScope.

Type:bool
visualisation_args

Configuration arguments used for glompo.core.scope.GloMPOScope if the optimization is being visualised dynamically.

Type:Dict[str, Any]
working_dir

Working directory in which all output files and directories are created. Note, the manager does not change the current working directory during the run.

Type:pathlib.Path
x0_generator

GloMPO object which returns a starting location for a new child optimizer. Can be based on previous results delivered by other optimizers.

Type:BaseGenerator
is_initialised

Returns True if this GloMPOManager instance has been initialised. Multiple initialisations are not allowed.

classmethod new_manager(*args, **kwargs) → glompo.core.manager.GloMPOManager[source]

Class method wrapper around setup() to directly initialise a new manager instance.

classmethod load_manager(*args, **kwargs) → glompo.core.manager.GloMPOManager[source]

Class method wrapper around load_checkpoint() to directly initialise a manager from a checkpoint.

setup(task: Callable[Sequence[float], float], bounds: Sequence[Tuple[float, float]], opt_selector: glompo.opt_selectors.baseselector.BaseSelector, working_dir: Union[pathlib.Path, str] = '.', overwrite_existing: bool = False, max_jobs: Optional[int] = None, backend: str = 'processes', convergence_checker: Optional[glompo.convergence.basechecker.BaseChecker] = None, x0_generator: Optional[glompo.generators.basegenerator.BaseGenerator] = None, killing_conditions: Optional[glompo.hunters.basehunter.BaseHunter] = None, share_best_solutions: bool = False, hunt_frequency: int = 100, status_frequency: int = 600, checkpoint_control: Optional[glompo.core.checkpointing.CheckpointingControl] = None, summary_files: int = 0, is_log_detailed: bool = False, visualisation: bool = False, visualisation_args: Optional[Dict[str, Any]] = None, force_terminations_after: int = -1, aggressive_kill: bool = False, end_timeout: Optional[int] = None, split_printstreams: bool = True)[source]

Generates the environment for a new globally managed parallel optimization job.

Parameters:
  • task – Function to be minimized. Accepts a 1D sequence of parameter values and returns a single value.
  • bounds – Sequence of tuples of the form (min, max) limiting the range of each parameter.
  • opt_selector – Selection criteria for new optimizers.
  • working_dir – If provided, GloMPO wil redirect its outputs to the given directory.
  • overwrite_existing – If True, GloMPO will overwrite existing files if any are found in the working_dir otherwise it will raise a FileExistsError if these results are detected.
  • max_jobs – The maximum number of threads the manager may create. Defaults to one less than the number of CPUs available to the system.
  • backend –

    Indicates the form of parallelism used by the optimizers.

    Accepts:

    'processes': Optimizers spawned as multiprocessing.Process

    'threads': Optimizers spawned as threading.Thread

    'processes_forced': Strongly discouraged, optimizers spawned as multiprocessing.Process and are themselves allowed to spawn multiprocessing.Process for function evaluations. See Parallelism for more details on this topic.

  • convergence_checker – Criteria used for convergence.
  • x0_generator – An instance of a subclass of BaseGenerator which produces starting points for the optimizer. If not provided, RandomGenerator is used.
  • killing_conditions – Criteria used for killing optimizers.
  • share_best_solutions – If True the manager will send the best ever seen solution to all its children whenever this is updated.
  • hunt_frequency – The number of function calls between successive attempts to evaluate optimizer performance and determine if they should be terminated.
  • status_frequency – Frequency (in seconds) with which status messages are logged.
  • checkpoint_control – If provided, the manager will use checkpointing during the optimization.
  • summary_files –

    Indicates what information the user would like saved to disk. Higher values also save all lower level information:

    1. Nothing is saved.
    2. YAML file with summary info about the optimization settings, performance and the result.
    3. PNG file showing the trajectories of the optimizers.
    4. HDF5 file containing iteration history for each optimizer.
  • is_log_detailed – If True the optimizers will call task.detailed_call and record the expanded return in the logs. Otherwise, optimizers will use task.__call__.
  • visualisation – If True then a dynamic plot is generated to demonstrate the performance of the optimizers. Further options (see visualisation_args) allow this plotting to be recorded and saved as a film.
  • visualisation_args – Optional arguments to parameterize the dynamic plotting feature. See GloMPO Scope.
  • force_terminations_after – If a value larger than zero is provided then GloMPO is allowed to force terminate optimizers that have either not provided results in the provided number of seconds or optimizers which were sent a kill signal have not shut themselves down within the provided number of seconds.
  • aggressive_kill – Ignored if backend is 'threads'. If True, child processes are forcibly terminated via SIGTERM. Else a termination message is sent to the optimizer to shut itself down. The latter option is preferred and safer, but there may be circumstances where child optimizers cannot handle such messages and have to be forcibly terminated.
  • end_timeout – The amount of time the manager will wait trying to smoothly join each child optimizer at the end of the run. Defaults to 10 seconds.
  • split_printstreams – If True, optimizer print messages will be intercepted and saved to separate files. See SplitOptimizerLogs

Notes

  1. To be process-safe task must be a standalone function which makes no modifications outside of itself. If this is not the case it is likely you would need to use a threaded backend.

  2. Do not use bounds to fix a parameter value as this will raise an error. Rather supply fixed parameter values through task_args or task_kwargs.

  3. An optimizer will not be started if the number of ‘slots’ it requires (i.e. BaseOptimizer.workers) will cause the total number of occupied ‘slots’ to exceed max_jobs, even if the manager is currently managing fewer than the number of jobs available. In other words, if the manager has registered a total of 30 of 32 slots filled, it will not start an optimizer that requires 3 or more slots.

  4. Checkpointing requires the use of the dill package for serialisation. If you attempt to checkpoint or supply checkpointing_controls without this package present, a warning will be raised and no checkpointing will occur.

  5. Caution

    Use force_terminations_after with caution as it runs the risk of corrupting the results queue, but ensures resources are not wasted on hanging processes.

  6. After end_timeout, if the optimizer is still alive and a process, GloMPO will send a terminate signal to force it to close. However, threads cannot be terminated in this way and the manager can leave dangling threads at the end of its routine. If the script ends after a GloMPO routine then all its children will be automatically garbage collected (provided 'processes_forced' backend has not been used).

    By default, this timeout is 10s if a process backend is used and infinite of a threaded backend is used. This is the cleanest approach for threads but can cause very long wait times or deadlocks if the optimizer does not respond to close signals and does not converge.

load_checkpoint(path: Union[pathlib.Path, str], task_loader: Optional[Callable[Union[pathlib.Path, str], Callable[Sequence[float], float]]] = None, task: Optional[Callable[Sequence[float], float]] = None, **glompo_kwargs)[source]

Initialise GloMPO from the provided checkpoint file and allows an optimization to resume from that point.

Parameters:
  • path – Path to GloMPO checkpoint file.
  • task_loader – Method to reconstruct task from files in the checkpoint.
  • task – In the case that the checkpoint does not contain a record of the task, it can be provided directly here.
  • **glompo_kwargs – Most arguments supplied to setup() can also be provided here. This will overwrite the values saved in the checkpoint. See Notes for arguments which cannot/should not be changed:

Notes

  1. When making a checkpoint, GloMPO attempts to persist the task directly. If this is not possible it will attempt to call checkpoint_save to produce some files into the checkpoint. task_loader is the function or method which can return a task from files within the checkpoint (see BaseFunction.checkpoint_load()).

    task_loader must accept a path to a directory containing the checkpoint files and return a callable which is the task itself.

    If both task_loader and task are provided, the manager will first attempt to use the task_loader and then only use task if that fails otherwise task is ignored.

  2. Caution

    GloMPO produces the requested log files when it closes (ie a convergence or crash). The working directory is, however, purged of old results at the start of the optimization (if overwriting is allowed). This behavior is the same regardless of whether the optimization is a resume or a fresh start. This means it is the user’s responsibility to save and move important files from the working_dir before a resume. This is particularly important for optimizer printstreams (which are overwritten) as well as movie files which can later be stitched together to make a single video of the entire optimization.

  3. GloMPO does not support making a single continuous recording of the optimization if it is stopped and resumed at some point. However, at the end of each section a movie file is made and these can be stitched together to make a continuous recording.

  4. The following arguments cannot/should not be sent to glompo_kwargs:

    bounds

    Many optimizers save the bounds during checkpointing. If changed here old optimizers will retain the old bounds but new optimizers will start in new bounds.

    max_jobs

    If this is decreased and falls below the number required by the optimizers in the checkpoint, the manager will attempt to adjust the workers for each optimizer to fit the new limit. Slots are apportioned equally (regardless of the distribution in the checkpoint) and there is no guarantee that the optimizers will actually respond to this change.

    visualisation_args

    Due to the semantics of GloMPOScope construction, these arguments will not be accepted by the loaded scope object.

    working_dir

    This can be changed, however, if a log file exists and you would like to append into this file, make sure to copy/move it to the new working_dir and name it 'glompo_log.h5' before loading the checkpoint otherwise GloMPO will create a new log file (see Outputs and Checkpointing).

start_manager() → glompo.common.namedtuples.Result[source]

Begins the optimization routine and returns the lowest encountered minimum.

checkpoint()[source]

Saves the state of the manager and any existing optimizers to disk. GloMPO can be loaded from these files and resume optimization from this state.

Notes

When checkpointing GloMPO will attempt to handle the task in three ways:

  1. pickle with the other manager variables, this is the easiest and most straightforward method.
  2. If the above fails, the manager will attempt to call task.checkpoint_save if it is present. This is expected to create file/s which is/are suitable for reconstruction during load_checkpoint(). When resuming a run the manager will attempt to reconstruct the task by calling the method passed to task_loader in load_checkpoint().
  3. If the manager cannot perform either of the above methods the checkpoint will be constructed without a task. In that case a fully initialised task must be given to load_checkpoint().
write_summary_file(dump_dir: Optional[pathlib.Path] = None)[source]

Writes a manager summary YAML file detailing the state of the optimization. Useful to extract output from a checkpoint.

Parameters:dump_dir – If provided, this will overwrite the manager working_dir allowing the output to be redirected to a different folder so as to not interfere with files in the working directory.