multiml.agent.metric module

Collection of pre-defined Metric classes.

class multiml.agent.metric.Metric

Bases: object

Abstraction class of metric calculation.

abstract calculate(): Return calculated metric.

class multiml.agent.metric.BaseMetric(storegate=None, pred_var_name='pred', true_var_name='true', var_names=None, phase='test', data_id=None)

Bases: Metric

Base class of metric calculation.

All Metric class need to inherit this BaseMetric class. Metric class is usually passed to agent class to calculate metric.

Examples

>>> metric = BaseMetric(storegate=storegate,
>>>                     pred_var_name='pred',
>>>                     true_var_name='true')
>>> metric.calculate()

__init__(storegate=None, pred_var_name='pred', true_var_name='true', var_names=None, phase='test', data_id=None)

Initialize the base class.

Parameters:

storegate (Storegate) – Storegate class instance.
pred_var_name (str) – name of variable for predicted values.
true_var_name (str) – name of variable for true values.
var_names (str) – ‘pred true’ variable names for shortcut.
phase (str) – ‘train’ or ‘valid’ or ‘test’ phase to calculate metric.
data_id (str) – specify data_id of storegate.

property storegate: Returns storegate of the base metric.

calculate()

Returns calculated metric.

Users need to implement algorithms.

property name: Returns name of metric.

property pred_var_name: Returns pred_var_name of metric,

property true_var_name: Returns true_var_name of metric.

property phase: Returns phase of metric.

property data_id: Returns data_id of metric.

property type

Return type of metric (e.f.

min).

get_true_pred_data()

Return true and pred data.

If special variable active is available, only samples with active is True are selected.

Returns:: true and pred values.
Return type:: ndarray, ndarray

class multiml.agent.metric.ZeroMetric(**kwargs)

Bases: BaseMetric

A dummy metric class to return always zero.

__init__(**kwargs): Initialize ZeroMetric.

calculate(): Returns zero.

class multiml.agent.metric.RandomMetric(**kwargs)

Bases: BaseMetric

A dummy metric class to return random value.

__init__(**kwargs): Initialize RandomMetric.

calculate(): Return random value.

class multiml.agent.metric.ValueMetric(**kwargs)

Bases: BaseMetric

A metric class to return a single value.

__init__(**kwargs): Initialize ValueMetric.

calculate(): Returns value.

class multiml.agent.metric.MSEMetric(**kwargs)

Bases: BaseMetric

A metric class to return Mean Square Error.

__init__(**kwargs): Initialize MSEMetric.

calculate(): Calculate MSE.

class multiml.agent.metric.ACCMetric(**kwargs)

Bases: BaseMetric

A metric class to return ACC.

__init__(**kwargs): Initialize ACCMetric.

calculate(): Calculate ACC.

class multiml.agent.metric.AUCMetric(**kwargs)

Bases: BaseMetric

A metric class to return AUC.

__init__(**kwargs): Initialize AUCMetric.

calculate(): Calculate AUC.