multiml.Saver

class multiml.Saver(save_dir=None, serial_id=None, mode='zarr', recreate=False)

Miscellaneous object management class.

Dictionary to save miscellaneous objects, and provides utility methods to manage ML metadata. There are two type of backends, zarr and dict, to store objects. zarr mode stores persistent objects, and dict mode stores temporary objects.

Examples

>>> from multiml import Saver
>>> saver = Saver()
>>> print(saver.save_dir)
>>> saver['key0'] = obj0
>>> saver['key1'] = obj1

__init__(save_dir=None, serial_id=None, mode='zarr', recreate=False)

Initialize Saver and create the base directory.

Parameters:

• save_dir (str) – directory path to save objects. If None is given, a temporary directory is created automatically by tempfile.mkdtemp() method.

• serial_id (int) – suffix of save_dir, i.e. save_dir.*serial_id*. If None is given, serial_id is incremented automatically based on existence of the directory.

• mode (str) – zarr or dict for default metadata management.

• recreate (bool) – recreate zarr database if True.

Methods

__init__([save_dir, serial_id, mode, recreate])	Initialize Saver and create the base directory.
add(key, obj[, mode, check])	Add object to given backend by key.
delete(key[, mode])	Delete key and object from the backends.
dump_ml(key[, suffix, ml_type])	Dump machine learning models and parameters.
init_zarr([recreate])	Initialize zarr database and confirm connection.
keys([mode])	Return registered keys in backends.
load_ml(key[, suffix])	Load machine learning models and parameters.
save()	Save the objects registered in dict to zarr.
set_mode(mode)	Set default database (backend) mode.
to_memory(key)	Move object from zarr to dict.
to_storage(key)	Move object from dict to storage.

Attributes

save_dir

Returns the name of base directory of Saver.

__init__(save_dir=None, serial_id=None, mode='zarr', recreate=False)

Initialize Saver and create the base directory.

Parameters:

• save_dir (str) – directory path to save objects. If None is given, a temporary directory is created automatically by tempfile.mkdtemp() method.

• serial_id (int) – suffix of save_dir, i.e. save_dir.*serial_id*. If None is given, serial_id is incremented automatically based on existence of the directory.

• mode (str) – zarr or dict for default metadata management.

• recreate (bool) – recreate zarr database if True.

__len__()

Returns the number of stored objects in zarr and dict.

Returns:

the total number of stored objects in zarr and dict.

Return type:

int

__setitem__(key, obj)

Set key and store object to the default backend.

key and obj are stored to the default backend, zarr or dict.

Parameters:

• key (str) – unique identifier of given object.

• obj (obj) – arbitrary object to be stored.

__getitem__(key)

Returns object for given key.

key is searched from the both zarr and dict backends regardless of the default backend mode.

Parameters:

key (str) – unique identifier to retrieve object.

Returns:

arbitrary object..

Return type:

obj

__delitem__(key)

Delete key and object from backends.

Parameters:

key (str) – unique identifier to be deleted.