multiml.hyperparameter module

Hyperparameter module.

class multiml.hyperparameter.Hyperparameter(name, data, is_continuous=False)

Bases: object

Hyperparameter management class.

This class describes hyperparameter names, type of parameters (continuous or discrete), and valid spaces. This class also retains the current value of hyperparameter, which is increased or decreased by step() method.

__init__(name, data, is_continuous=False)

Initialize Hyperparameter class.

Parameters:

name (str) – hyperparameter name.
data (tuple or list) – tuple must contain (min, max, default_step) for continuous mode, where min and max are the maximum and minimum value of the hyperparameter, and default_step indicates a interval of sampling (please see get_grid() method). list must contain valid parameter values for discrete mode.
is_continuous (bool) – hyperparameter is continuous or not.

Examples

>>> # continuous hyperparameter
>>> hp0 = Hyperparameter('hp0', (0., 1., 0.2), True)
>>> # discrete hyperparameter
>>> hp1 = Hyperparameter('hp1', [0., 0.2, 0.4, 0.6, 0.8. 1.0])

__len__(): Returns the number of possible hyperparameter values.

__getitem__(item)

Returns the hyperparameter value by index.

Parameters:: item (int) – index of get_grid() result.
Returns:: the hyperparameter value.
Return type:: int or float

property name

Returns the name of hyperparameter.

Returns:: the name of hyperparameter.
Return type:: str

property value

Returns the current value of hyperparameter.

Returns:: the current value of hyperparameter.
Return type:: int or float

property is_continuous

Hyperparameter is continuous mode or discrete mode.

Returns:: True if the hyperparameter is defined as continuous.
Return type:: bool

step(step=None)

Increase or decrease the current hyperparameter value.

Parameters:: step (int or float) – the hyperparameter value is increased or decreased according to sign of given step for continuous mode. Also the current index of hyperparameter is changed by step for discrete mode.
Returns:: if the value is changed or not.
Return type:: bool

set_min_value(): Set the minimum value to the current value.

set_max_value(): Set the maximum value to the current value.

get_grid()

Returns available values of the hyperparameter.

If the hyperparameter is continuous mode, values are sampled by step of default_step between min and max.

Returns:: list of hyperparameter values.
Return type:: list

class multiml.hyperparameter.Hyperparameters(hps=None)

Bases: object

Utility class to manage Hyperparameter classes.

The Hyperparameters class provides interfances to manage Hyperparameter class instances. This Hyperparameters class instance should be passed to TaskScheduler together with corresponding subtask.

Examples

>>> hps_dict = {
>>>     'hp_layers': [5, 10, 15, 20], # discrete
>>>     'hp_alpha': [1.0, 2.0, 3.0] # discrete
>>> }
>>> hps = Hyperparameters(hps_dict)
>>> hps.set_min_hps()
>>> hps.get_current_hps()
>>>  -> {'hp_layers': 5, 'hp_alpha': 1.0}
>>> hps.step('hp_layers', 2)
>>> hps.get_current_hps()
>>>  -> {'hp_layers': 15, 'hp_alpha': 1.0}
>>> hps.step('hp_alpha', 1)
>>> hps.get_current_hps()
>>>  -> {'hp_layers': 15, 'hp_alpha': 2.0}

__init__(hps=None)

Initialize Hyperparameters class.

hps option provides a shortcut to register hyperparameters. This option works only for discrete hyperparameters for now.

Parameters:: hps (dict) – a dictionary of hyperparameters. Please see add_hp_from_dict() method.

__len__(): Returns the number of all possible combinations of hyperparameters.

__getitem__(item)

Returns registered Hyperparameter class instance by index.

If item is str, Hyperparameter is searched by its name, and class instance is returned if it exists. If item is int, a dictionary of selected hyperparameters from get_grid_hps() method is returned.

Parameters:: item (str or int) – the name of hyperparameter, or index of all possible combination from get_grid_hps() method.
Returns:: please see the above description.
Return type:: Hyperparameter or dict

__contains__(item)

Check if Hyperparameter is registered or not.

Parameters:: item (str) – the name of Hyperparameter.
Returns:: True if Hyperparameter exists.
Return type:: bool

step(hp_name=None, step=None)

Update the current hyperparameter values.

Parameters:

hp_name (str) – name of hyperparameter. If given np_name is None, any updatable hp_name is selected arbitrarily.
step (int or float) – see Hyperparameter class.

Returns:

if the value is changed or not.

Return type:

bool

add_hp_from_dict(hps)

Add hyperparameters from dictionary.

Values of the dictionary should be a list of allowed hyperparameter values. Continuous mode is not supported yet.

Parameters:: hps (dict) – dict of hyperparameter values.

Examples

>>> hp_dict = dict(hp0=[0, 1, 2], hp1=[3, 4, 5])
>>> hps.add_hp_from_dict(hp_dict)

add_hp(name, values, is_continuous=False)

Add hyperparameter.

Parameters:

name (str) – the name of hyperparameter.
values (tuple or list) – please see Hyperparameter class.
is_continuous (bool) – Hyperparameter is continuous or not.

set_min_hps(): Set the minimum value for each Hyperparameter.

set_max_hps(): Set the maximum value for each Hyperparameter.

get_current_hps()

Returns the current values of Hyperparameters.

Returns:: dict of the current hyperparameter values.
Return type:: dict

get_grid_hps()

Returns all possible combination of hyperparameters values.

If registered Hyperparameter class instance is continuous mode, values are sampled between min and max by dividing default step.

Returns:: all possible combination of hyperparameters values.
Return type:: list

get_hp_names()

Returns the names of hyperparameters.

Returns:: the names of hyperparameters.
Return type:: list