Helpers

Useful static functions and classes used throughout the GloMPO package.

class glompo.common.helpers.SplitOptimizerLogs(filepath: Union[pathlib.Path, str] = '', propagate: bool = False, formatter: Optional[logging.Formatter] = None)

Splits print statements from child processes and threads to text files. If this filter is applied to a Handler on the 'glompo.optimizers' logger it will automatically separate the single 'glompo.optimizers' logging stream into separate ones for each individual optimizer.

Parameters:

• filepath – Directory in which new log files will be located.
• propagate – If propagate is True then the filter will allow the message to pass through the filter allowing all 'glompo.optimizers' logging to be simultaneously recorded together.
• formatter – Formatting to be applied in the new logs. If not supplied the logging default is used.

Examples

>>> frmttr = logging.Formatter("%(levelname)s : %(name)s : %(processName)s :: %(message)s")

Adds individual handlers for each optimizer created. Format for the new handlers is set by frmttr propagate=True sends the message on to opt_handler which in this case is sys.stdout.

>>> opt_filter = SplitOptimizerLogs("diverted_logs", propagate=True, formatter=frmttr)
>>> opt_handler = logging.StreamHandler(sys.stdout)
>>> opt_handler.addFilter(opt_filter)
>>> opt_handler.setFormatter(frmttr)

Messages of the 'INFO' level will propogate to sys.stdout.

>>> opt_handler.setLevel('INFO')
>>> logging.getLogger("glompo.optimizers").addHandler(opt_handler)

The level for the handlers made in SplitOptimizerLogs is set at the higher level. Here 'DEBUG' level messages will be logged to the files even though 'INFO' level propagates to the console.

>>> logging.getLogger("glompo.optimizers").setLevel('DEBUG')

filter(record: logging.LogRecord) → int

Determine if the specified record is to be logged.

Is the specified record to be logged? Returns 0 for no, nonzero for yes. If deemed appropriate, the record may be modified in-place.

glompo.common.helpers.nested_string_formatting(nested_str: str) → str

Reformat strings produced by the ._CombiCore class. BaseHunters and BaseCheckers produce strings detailing their last evaluation. This method parses and indents each nested level of the string to make it more human readable.

Parameters:nested_str – Return produced by BaseHunter.__str__() or BaseHunter.str_with_result(). Returns:String with added nesting and indenting. Return type:str

Examples

>>> nested_string_formatting("[TrueHunter() AND\n"
...                          "[[TrueHunter() OR\n"
...                          "[FalseHunter() AND\n"
...                          "[TrueHunter() OR\n"
...                          "FalseHunter()]]]\n"
...                          "OR\n"
...                          "FalseHunter()]]")
"TrueHunter() AND\n" \
"[\n" \
" [\n" \
"  TrueHunter() OR\n" \
"  [\n" \
"   FalseHunter() AND\n" \
"   [\n" \
"    TrueHunter() OR\n" \
"    FalseHunter()\n" \
"   ]\n" \
"  ]\n" \
" ]\n" \
" OR\n" \
" FalseHunter()\n" \
"]"

glompo.common.helpers.is_bounds_valid(bounds: Sequence[Tuple[float, float]], raise_invalid=True) → bool

Checks if provided parameter bounds are valid. ‘Valid’ is defined as meaning that every lower bound is less than the upper bound and every bound is finite.

Parameters:

• bounds – Sequence of min/max pairs indicating the interval in which the optimizer must search for each parameter.
• raise_invalid – If True raises an error if the bounds are invalid otherwise a bool is returned.

Returns:

True if the bounds are all valid, False otherwise.

Return type:

bool

Raises:

ValueError – If raise_invalid is True and bounds are invalid.

Examples

>>> is_bounds_valid([(0, 1), (-1, 0)])
True

>>> is_bounds_valid([(0, 0), (0, float('inf'))], False)
False

glompo.common.helpers.distance(pt1: Sequence[float], pt2: Sequence[float])

Calculate the Euclidean distance between two points.

Examples

>>> distance([0,0,0], [1,1,1])
1.7320508075688772

glompo.common.helpers.glompo_colors(opt_id: Optional[int] = None) → Union[matplotlib.colors.ListedColormap, Tuple[float, float, float, float]]

Returns a matplotlib.colors.ListedColormap containing the custom GloMPO color cycle. If opt_id is provided than the specific color at that index is returned instead.

glompo.common.helpers.present_memory(bytes_: float, digits: int = 2) → str

Accepts an integer number of bytes and returns a string formatted to the most appropriate units.

Parameters:

• bytes – Number of bytes to write in human readable format
• digits – Number of decimal places to include in the result

Returns:

Converted data quantity and units

Return type:

str

Examples

>>> present_memory(123456789, 1)
'117.7MB'

glompo.common.helpers.rolling_min(x: Sequence[float]) → Sequence[float]

Returns a vector of shape x where each index has been replaced by the smallest number seen thus far when reading the list sequentially from left to right.

Examples

>>> rolling_min([3, 4, 5, 6, 2, 3, 4, 1, 2, 3])
[3, 3, 3, 3, 2, 2, 2, 1, 1, 1]

glompo.common.helpers.unravel(seq: Union[Any, Sequence[Any]]) → Iterator[str]

From a nested sequence of items of any type, return a flatten sequence of items.

Examples

>>> unravel([0, [1], [2, 3, [4, 5, [6], 7], 8, [9]]])
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

glompo.common.helpers.infer_headers(infer_from: Sequence[Any]) → Dict[str, tables.description.Col]

Infers tables.Col types based on a function evaluation return. Only used if BaseFunction.headers() is not defined. The produced headings are required by FileLogger. Default header names will be used: 'result_0', ..., 'result_N'.

Parameters:infer_from – A function evaluation result Returns:Mapping of column names to type Return type:Dict[str, tables.Col]

Examples

>>> import tables
>>> infer_headers((1232.1431423, [213, 345, 675], False, "valid"))
{'result_0': tables.Float64Col(pos=0),
 'result_1': tables.Int64Col((1, 3), pos=1),
 'result_2': tables.BoolCol(pos=2),
 'result_3': tables.StringCol(280, pos=3)}

glompo.common.helpers.deepsizeof(obj) → int

Recursively determines the byte size of an object. Any initialised objects (not including Python primitives) must correct implement __sizeof__() for this method to work correctly.

class glompo.common.helpers.LiteralWrapper

Used by YAML to save some block strings as literals

class glompo.common.helpers.FlowList

Used to wrap lists which should appear in YAML flow style rather than default block style.

class glompo.common.helpers.BoundGroup

Used to better represent the parameter bounds in a human readable but reusable way in YAML.

glompo.common.helpers.literal_presenter(dumper: yaml.dumper.Dumper, data: str)

Wrapper around string for better readability in YAML file.

glompo.common.helpers.optimizer_selector_presenter(dumper, opt_selector: BaseSelector)

Unique YAML constructor for the BaseSelector.

glompo.common.helpers.generator_presenter(dumper, generator: BaseGenerator)

Unique YAML constructor for the BaseGenerator.

glompo.common.helpers.flow_presenter(dumper, lst)

YAML Presenter for a FlowList style list.

glompo.common.helpers.numpy_dtype_presenter(dumper, numpy_type)

Unique YAML constructor for numpy.dtype.

glompo.common.helpers.numpy_array_presenter(dumper, numpy_arr)

Unique YAML constructor for numpy.ndarray.

glompo.common.helpers.bound_group_presenter(dumper, bound_group)

Unique YAML constructor for Bound.

glompo.common.helpers.unknown_object_presenter(dumper, unknown_class: object)

Parses all remaining classes into strings and python primitives for YAML files. To ensure the YAML file is human readable and can be loaded only with native python types. This constructor parses all unknown objects into a dictionary containing their name and instance variables or, if uninitialised, just the class name.

class glompo.common.helpers.WorkInDirectory(path: Union[pathlib.Path, str])

Context manager to manage the creation of new files in a different directory from the working one.

Parameters:path – A directory to which the working directory will be changed on entering the context manager. If the directory does not exist, it will be created. The working directory is changed back on exiting the context manager.

exception glompo.common.helpers.CheckpointingError

Error raised during creation of a checkpoint which would result in an incomplete checkpoint.