multiml.task.keras.modules package

Submodules

Module contents

class multiml.task.keras.modules.BaseModel(*args, **kwargs)

Bases: Model

__init__(*args, **kwargs)

Base model to overwrite train_step().

set_pred_index(pred_index)
train_step(data)

The logic for one training step.

This method can be overridden to support custom training logic. This method is called by Model.make_train_function.

This method should contain the mathematical logic for one step of training. This typically includes the forward pass, loss calculation, backpropagation, and metric updates.

Configuration details for how this logic is run (e.g. tf.function and tf.distribute.Strategy settings), should be left to Model.make_train_function, which can also be overridden.

Parameters:

data – A nested structure of `Tensor`s.

Returns:

A dict containing values that will be passed to tf.keras.callbacks.CallbackList.on_train_batch_end. Typically, the values of the Model’s metrics are returned. Example: {‘loss’: 0.2, ‘accuracy’: 0.7}.

test_step(data)

The logic for one evaluation step.

This method can be overridden to support custom evaluation logic. This method is called by Model.make_test_function.

This function should contain the mathematical logic for one step of evaluation. This typically includes the forward pass, loss calculation, and metrics updates.

Configuration details for how this logic is run (e.g. tf.function and tf.distribute.Strategy settings), should be left to Model.make_test_function, which can also be overridden.

Parameters:

data – A nested structure of `Tensor`s.

Returns:

A dict containing values that will be passed to tf.keras.callbacks.CallbackList.on_train_batch_end. Typically, the values of the Model’s metrics are returned.

select_pred_data(y_pred)
class multiml.task.keras.modules.FunctionalModel(*args, **kwargs)

Bases: Functional

__init__(*args, **kwargs)

Base model to overwrite train_step().

TODO: this class is to avoid mix of functional API and subclass.

set_pred_index(pred_index)
train_step(data)

The logic for one training step.

This method can be overridden to support custom training logic. This method is called by Model.make_train_function.

This method should contain the mathematical logic for one step of training. This typically includes the forward pass, loss calculation, backpropagation, and metric updates.

Configuration details for how this logic is run (e.g. tf.function and tf.distribute.Strategy settings), should be left to Model.make_train_function, which can also be overridden.

Parameters:

data – A nested structure of `Tensor`s.

Returns:

A dict containing values that will be passed to tf.keras.callbacks.CallbackList.on_train_batch_end. Typically, the values of the Model’s metrics are returned. Example: {‘loss’: 0.2, ‘accuracy’: 0.7}.

test_step(data)

The logic for one evaluation step.

This method can be overridden to support custom evaluation logic. This method is called by Model.make_test_function.

This function should contain the mathematical logic for one step of evaluation. This typically includes the forward pass, loss calculation, and metrics updates.

Configuration details for how this logic is run (e.g. tf.function and tf.distribute.Strategy settings), should be left to Model.make_test_function, which can also be overridden.

Parameters:

data – A nested structure of `Tensor`s.

Returns:

A dict containing values that will be passed to tf.keras.callbacks.CallbackList.on_train_batch_end. Typically, the values of the Model’s metrics are returned.

select_pred_data(y_pred)
class multiml.task.keras.modules.SoftMaxDenseLayer(*args, **kwargs)

Bases: Layer

__init__(kernel_initializer='zeros', kernel_regularizer=None, dropout_rate=None, **kwargs)

Constructor.

Parameters:

  • kernel_initializer (str) – initializer for softmax weights

  • kernel_regularizer (str) – regularizer for softmax weights

  • dropout_rate (float) – dropout rate

build(input_shape)

Creates the variables of the layer (optional, for subclass implementers).

This is a method that implementers of subclasses of Layer or Model can override if they need a state-creation step in-between layer instantiation and layer call.

This is typically used to create the weights of Layer subclasses.

Parameters:

input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

call(inputs, training=None)

This is where the layer’s logic lives.

Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.

Parameters:

  • inputs – Input tensor, or list/tuple of input tensors.

  • **kwargs – Additional keyword arguments. Currently unused.

Returns:

A tensor or list/tuple of tensors.

get_config()

Returns the config of the layer.

A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.

The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).

Returns:

Python dictionary.

class multiml.task.keras.modules.MLPBlock(*args, **kwargs)

Bases: Model

__init__(layers=None, activation=None, activation_last=None, kernel_regularizer=None, bias_regularizer=None, batch_norm=False, *args, **kwargs)

Constructor.

Parameters:

  • layers (list) – list of hidden layers

  • activation (str) – activation function for MLP

  • activation_last (str) – activation function for the MLP last layer

  • batch_norm (bool) – use batch normalization

  • kernel_regularizer (str) – kernel regularizer

  • bias_regularizer (str) – bias regularizer

  • *args – Variable length argument list

  • **kwargs – Arbitrary keyword arguments

call(input_tensor, training=False)

Calls the model on new inputs.

In this case call just reapplies all ops in the graph to the new inputs (e.g. build a new computational graph from the provided inputs).

Parameters:

  • inputs – A tensor or list of tensors.

  • training – Boolean or boolean scalar tensor, indicating whether to run the Network in training mode or inference mode.

  • mask – A mask or list of masks. A mask can be either a tensor or None (no mask).

Returns:

A tensor if there is a single output, or a list of tensors if there are more than one outputs.

class multiml.task.keras.modules.Conv2DBlock(*args, **kwargs)

Bases: Model

__init__(layers_conv2d=None, conv2d_padding='valid', *args, **kwargs)

Constructor.

Parameters:

  • layers_conv2d (list(tuple(str, dict))) – configs of conv2d layer. list of tuple(op_name, op_args).

  • conv2d_padding (str) – padding option of conv2d (valid or same)

  • *args – Variable length argument list

  • **kwargs – Arbitrary keyword arguments

call(input_tensor, training=False)

Calls the model on new inputs.

In this case call just reapplies all ops in the graph to the new inputs (e.g. build a new computational graph from the provided inputs).

Parameters:

  • inputs – A tensor or list of tensors.

  • training – Boolean or boolean scalar tensor, indicating whether to run the Network in training mode or inference mode.

  • mask – A mask or list of masks. A mask can be either a tensor or None (no mask).

Returns:

A tensor if there is a single output, or a list of tensors if there are more than one outputs.

class multiml.task.keras.modules.ConnectionModel(*args, **kwargs)

Bases: ConnectionModel, BaseModel

__init__(*args, **kwargs)
Parameters:

  • *args – Variable length argument list

  • **kwargs – Arbitrary keyword arguments

call(inputs)

Calls the model on new inputs.

In this case call just reapplies all ops in the graph to the new inputs (e.g. build a new computational graph from the provided inputs).

Parameters:

  • inputs – A tensor or list of tensors.

  • training – Boolean or boolean scalar tensor, indicating whether to run the Network in training mode or inference mode.

  • mask – A mask or list of masks. A mask can be either a tensor or None (no mask).

Returns:

A tensor if there is a single output, or a list of tensors if there are more than one outputs.

class multiml.task.keras.modules.DARTSModel(*args, **kwargs)

Bases: ConnectionModel

__init__(optimizer_alpha, optimizer_weight, learning_rate_alpha, learning_rate_weight, zeta, *args, **kwargs)

Constructor.

Parameters:

  • optimizer_alpha (str) – optimizer for alpha in DARTS optimization

  • optimizer_weight (str) – optimizer for weight in DARTS optimization

  • learning_rate_alpha (float) – learning rate (epsilon) for alpha in DARTS optimization

  • learning_rate_weight (float) – learning rate (epsilon) for weight in DARTS optimization

  • zeta (float) – zeta parameter in DARTS optimization

train_step(data)

The logic for one training step.

This method can be overridden to support custom training logic. This method is called by Model.make_train_function.

This method should contain the mathematical logic for one step of training. This typically includes the forward pass, loss calculation, backpropagation, and metric updates.

Configuration details for how this logic is run (e.g. tf.function and tf.distribute.Strategy settings), should be left to Model.make_train_function, which can also be overridden.

Parameters:

data – A nested structure of `Tensor`s.

Returns:

A dict containing values that will be passed to tf.keras.callbacks.CallbackList.on_train_batch_end. Typically, the values of the Model’s metrics are returned. Example: {‘loss’: 0.2, ‘accuracy’: 0.7}.

test_step(data)

The logic for one evaluation step.

This method can be overridden to support custom evaluation logic. This method is called by Model.make_test_function.

This function should contain the mathematical logic for one step of evaluation. This typically includes the forward pass, loss calculation, and metrics updates.

Configuration details for how this logic is run (e.g. tf.function and tf.distribute.Strategy settings), should be left to Model.make_test_function, which can also be overridden.

Parameters:

data – A nested structure of `Tensor`s.

Returns:

A dict containing values that will be passed to tf.keras.callbacks.CallbackList.on_train_batch_end. Typically, the values of the Model’s metrics are returned.

get_index_of_best_submodels()
class multiml.task.keras.modules.SumTensor(*args, **kwargs)

Bases: MeanTensor

result()

Computes and returns the metric value tensor.

Result computation is an idempotent operation that simply calculates the metric value using the state variables.

class multiml.task.keras.modules.EnsembleModel(*args, **kwargs)

Bases: Model

__init__(models, prefix, ensemble_type, dropout_rate=None, individual_loss=False, *args, **kwargs)

Constructor.

Parameters:

  • models (list(tf.keras.Model)) – list of keras models for ensembling

  • prefix (str) – prefix for a layer’s name

  • ensemble_type (str) – type of ensemble way (linear or softmax)

  • dropout_rate (float) – dropout rate. Valid only for ensemble_type = softmax

  • individual_loss (bool) – use multiple outputs

call(inputs, training=False)

Calls the model on new inputs.

In this case call just reapplies all ops in the graph to the new inputs (e.g. build a new computational graph from the provided inputs).

Parameters:

  • inputs – A tensor or list of tensors.

  • training – Boolean or boolean scalar tensor, indicating whether to run the Network in training mode or inference mode.

  • mask – A mask or list of masks. A mask can be either a tensor or None (no mask).

Returns:

A tensor if there is a single output, or a list of tensors if there are more than one outputs.