thread_chunks.Checkpoint

class thread_chunks.Checkpoint(*args, **kwargs)

Bases: SaveableObject

A data structure to store the progress of the parallel execution of a function func.

Attributes

name	The file name of the object (without the file extension).
path	The current file path of the object.
func_name	The name of the function being executed in parallel.
parameters	A list of argument lists for the function.
output	An ordered list of the outputs of the function: output[i]=func_name(*parameters[i]).
completed	A list of which outputs have been computed.
index	The largest index of the parameter list currently being computed.
done	The number of outputs already computed.

Methods

__init__	Initialises a data structure to store the progress of the parallel execution of a function.
load	Loads a pickled instance.
loadif	Attempts to load from a specified path.
loadifparams	Attempts to load() from a specified path.
save	Pickles the current instance.
tryload	Attempts to load() from the specified path.
update_save	Pickles the current instance and retains the saved arguments if they exist.

classmethod load(path: str, new_path: str | None = None, strict_typing: bool = True) → SaveableObject

Loads a pickled instance.

Parameters:

• path (str) – The path of the pickle.

• new_path (str, optional) – The path to replace the previous path with. If None the path is not replaced. By default None.

• strict_typing (bool, optional) – If True then the loaded instance must be an instance of cls. By default True.

Returns:

The loaded instance.

Return type:

SaveableObject

Raises:

TypeError – If strict_typing and the loaded instance is not an instance of cls.

Notes

strict_typing=True acts as a safety guard. Setting strict_typing=False may increase the probability of unexpected or uncaught errors.

classmethod loadif(*args, **kwargs) → Tuple[SaveableObject, bool]

Attempts to load from a specified path. If the loading fails or no path is specified then a new instance of the object is generated with the specified *args and **kwargs.

Parameters:

• *args – The arguments to pass to the initialisation on a failed load().

• path (str, optional) – The path of the pickle, by default the parameter is not specified.

• **kwargs – The keyword arguments to pass to the initialisation on a failed load().

Returns:

The loaded or initialised instance followed by True if the instance was loaded and False if the instance was initialised.

Return type:

(SaveableObject, bool)

classmethod loadifparams(*args, dependencies: dict = {}, **kwargs) → Tuple[SaveableObject, bool]

Attempts to load() from a specified path. If the loading fails or no path is specified or the parameters do not match the saved parameters then a new instance of the object is generated with the specified *args and **kwargs.

Parameters:

• *args – The arguments to pass to the initialisation on a failed load().

• path (str, optional) – The path of the pickle, by default the parameter is not specified.

• dependencies (dict, optional, must be specified as a keyword argument) – A dictionary of additional dependencies to check.

• **kwargs – The keyword arguments to pass to the initialisation on a failed load().

Returns:

The loaded or initialised instance followed by True if the instance was loaded and False if the instance was initialised.

Return type:

(SaveableObject, bool)

classmethod tryload(path: str | None, new_path: str | None = None, strict_typing: bool = True) → SaveableObject | Literal[False]

Attempts to load() from the specified path. If the loading fails then False is returned.

Parameters:

• path (str, optional) – The path of the pickle. If None then False is returned.

• new_path (str, optional) – The path to replace the previous path with. If None the path is not replaced. By default None.

• strict_typing (bool, optional) – If True then the loaded instance must be an instance of cls. By default True.

Returns:

If succeeded the loaded instance, else False.

Return type:

SaveableObject | Literal[False]

Notes

strict_typing=True acts as a safety guard. Setting strict_typing=False may increase the probability of unexpected or uncaught errors.

__init__(func_name: str, parameters: List[List[Any]], output: List[Any], completed: List[bool], index: int, done: int, path: str | None = None)

Initialises a data structure to store the progress of the parallel execution of a function.

Parameters:

• func_name (str) – The name of the function being executed in parallel.

• parameters (List[List[Any]]) – A list of argument lists for the function. The i`th call of the function is ``func_name(*parameters[i])`.

• output (List[Any]) – An ordered list of the outputs of the function: output[i]=func_name(*parameters[i]). output[i] should be set to None if func_name(*parameters[i]) is yet to finish execution.

• completed (List[bool]) – A list of which outputs have been computed. True represents the ith output has been computed.

• index (int) – The largest index of the parameter list currently being computed.

• done (int) – The number of outputs already computed.

• path (str, optional) – File path to save the object to. If None then the object is not saved. By default None.

save(path: str | None = None)

Pickles the current instance.

Parameters:

path (str, optional) – The path to pickle the instance to. If None is specified then the attribute path is used instead. By default None.

Raises:

ValueError – Raised if no path specified either by the parameter path or the attribute path.

Notes

If no file extension is provided then the class name and the .pkl extension are appended to the file name.

update_save(path: str | None = None) → bool

Pickles the current instance and retains the saved arguments if they exist.

Parameters:

path (str, optional) – The path to pickle the instance to. If None is specified then the attribute path is used instead. By default None.

Returns:

True if there was an argument pickle to retain. False if there was not an argument pickle to retain.

Return type:

bool

Raises:

ValueError – Raised if no path specified.

Notes

If no file extension is provided then the class name and the .pkl extension are appended to the file name.

completed: List[bool]

A list of which outputs have been computed. True represents the ith output has been computed.

done: int

The number of outputs already computed.

func_name: str

The name of the function being executed in parallel.

index: int

The largest index of the parameter list currently being computed.

property name: str | None

The file name of the object (without the file extension). Note that name is read only.

output: List[Any]

An ordered list of the outputs of the function: output[i]=func_name(*parameters[i]). output[i] should be set to None if func_name(*parameters[i]) is yet to finish execution.

parameters: List[List[Any]]

A list of argument lists for the function. The i th call of the function is func_name(*parameters[i]).

property path: str | None

The current file path of the object.

Notes

On setting the value, if no file extension is provided then the class name and the .pkl extension are appended to the file name.