Epoch results

Summary

Results of tuning using an epoch trainer.

TuningResults has many Training objects, each representing training for a one letstune.Params object.

Each Training has many Epoch objects, each representing training for a one epoch.

Metric value

In context of TuningResults and Training objects, metric value means metric value from the best epoch (not last!).

Reporting

TuningResults provides useful reporting utilities, like dataframe export.

The dataframe can be used for further analysis and visualization.

Model loading

To unpickle the best model:

chk = tuning[0].best_epoch.checkpoint
model = chk.load_pickle()

Tuning results class

class letstune.results.epoch.TuningResults

Results of epoch training.

List of Training objects, each representing one letstune.Params.

Bases: typing.Generic [P], collections.abc.Sequence [letstune.results.epoch.Training [P]]

__getitem__(i: int) → letstune.results.epoch.Training[P]: Get i-th best training. tuning[0] gives the best training.

__getitem__(slice: slice) → list[letstune.results.epoch.Training[P]]: Get slice of top trainings. tuning[:5] gives 5 best trainings.

__len__() → int: Get number of trainings.

property errors: collections.abc.Sequence[letstune.results.epoch.Error]: Sequence of failed trainings.

property metric: str: Metric used in the tuning.

to_df() → DataFrame

Get dataframe describing all trainings in the tuning.

Columns correspond to fields from training objects.

Other classes

class letstune.results.epoch.Training

Training of a model for given letstune.Params.

List of Epoch objects, each representing one epoch.

Bases: typing.Generic [ P ], collections.abc.Sequence [ letstune.results.epoch.Epoch ]

Basic data

training_id: int

params: P

round: int: Round survived by the training. Calculated using metric_value.

Epochs

best_epoch: Epoch

The best epoch in the training.

Can be different than last_epoch due to overfit.

last_epoch: Epoch: Last epoch in the training.

__getitem__(i: int) → letstune.results.epoch.Epoch: Get i-th epoch.

__len__() → int: Get number of epochs.

Summarized metric values

property metric_value: float: Metric value of the best epoch.

Summarized time properties

property start_time: datetime: Start time of the first epoch.

property end_time: datetime: End time of the last epoch.

property duration: timedelta: Total duration of all epochs.

class letstune.results.epoch.Epoch

Single epoch from Training.

Basic data

training_id: int

epoch_id: int

Metric values

metric_values: MappingProxyType[str, float]

metric_value: float

total_metric_value: float: The best metric value up to this epoch in this training.

Time

start_time: datetime

end_time: datetime

property duration: timedelta: Time elapsed from start_time to end_time.

total_duration: timedelta: Total training duration up to this epoch.

Checkpoint

property checkpoint: Any: Checkpoint with fitted model during this training.

class letstune.results.epoch.Error(*, training_id: int, params: str, msg: str)

training_id: int

params: str

msg: str