blocks.hooks

Classes

• ComputeCTCStats: Accumulate CTC labels, labels_mask and predictions in order to compute sentence error rate after each epoch.

• ComputeCTCStatsWithEOS: Same as ComputeCTCStats but use decode_ctc_prediction_with_eos() for decoding.

• SaveImages: Save images of the provided streams.

• VisualizeMasks: Join and save images with the corresponding segmentation masks after each batch.

• VisualizeMasksGT: Join and save image with dyed segmentation mask prediction.

• VisualizeRectangleProposals: Visualize proposed rectangle regions.

• ComputeRegionProposals: emloop hook interface.

• ComputeRegionStats: Compute rectangle RPN stats (recall and overlap ratio).

• VisualizeThresholdedPrediction: Save images overlayed by their semi-transparent prediction masks for each of the given thresholds. The entire batch is saved as a single image matrix where

• Threshold: Threshold image.

• SaveNumpy: Dump numpy arrays.

class blocks.hooks.ComputeCTCStats(labels='labels', predictions='predictions', labels_mask='labels_mask', decode_fn=<function ComputeCTCStats.<lambda>>, **kwargs)

Bases: emloop.hooks.accumulate_variables.AccumulateVariables

Accumulate CTC labels, labels_mask and predictions in order to compute sentence error rate after each epoch.

• labels: 2d array of zero-padded target labels

• labels_mask: 2d array of labels masks wherein ones mark the valid values

• predictions: 3d array of predictions of shape (batch x position x probs) wherein probs have dim num_classes + 1

compute ctc stats for each epoch

hooks:
  - blocks.hooks.ComputeCTCStats

__init__(labels='labels', predictions='predictions', labels_mask='labels_mask', decode_fn=<function ComputeCTCStats.<lambda>>, **kwargs)

Create new ComputeCTCStats hook.

Parameters

• labels (str) – labels variable name

• predictions (str) – predictions variable name

• labels_mask (str) – labels_mask variable name

• decode_fn – function for decoding raw CTC output, decode_ctc_prediction() by default

static _edit_distance(s1, s2)

Compute Levenshtein edit distance between two strings.

after_epoch(epoch_data, **kwargs)

Compute sentence error rate and mean edit distance for each stream and save it to the epoch_data as sentence_error and edit_distance, respectively.

Parameters

epoch_data – epoch data

Return type

None

class blocks.hooks.ComputeCTCStatsWithEOS(**kwargs)

Bases: blocks.hooks.compute_ctc_stats.ComputeCTCStats

Same as ComputeCTCStats but use decode_ctc_prediction_with_eos() for decoding.

__init__(**kwargs)

Create new ComputeCTCStatsWithEOS hook.

class blocks.hooks.SaveImages(output_dir, streams=['train'], variable='images', id_variable='ids', out_format='png', factor=1, img_count=None, batch_count=None, **kwargs)

Bases: hooks.AbstractHook

Save images of the provided streams.

This hook can be used as a base class for the hooks, which need systematically save their results. For this purpose, process_img method and image_suffix property should be overridden.

Caution

Saving all the images may require considerable amount of time. Use this hook only for debugging or limit the number of saved images with image_count or batch_count parameters.

Save images from test and valid streams, stored in variable x.

hooks:
  - blocks.hooks.SaveImages:
      variable: x
      streams: [test, valid]

Save the first two images from the first ten batches (from the train stream).

hooks:
  - blocks.hooks.SaveImages:
      img_count: 2
      batch_count: 10

__init__(output_dir, streams=['train'], variable='images', id_variable='ids', out_format='png', factor=1, img_count=None, batch_count=None, **kwargs)

Parameters

• output_dir (str) – output directory where images will be saved

• streams (Iterable[str]) – list of stream names to be dumped

• variable (str) – name of the variable representing the source image

• id_variable (str) – name of the variable which represents a unique example id

• out_format (str) – extension of the saved image

• factor (float) – a constant by which the image is multiplied

• img_count (Optional[int]) – count of images which will be saved from each batch (first img_count images will be saved)

• batch_count (Optional[int]) – count of batches from which the images will be saved (first batch_count will be processed)

_reset()

Reset _batch_count to initial value.

Return type

None

after_batch(stream_name, batch_data)

Save images in provided streams from selected variable. The amount of batches and images to be processed is possible to control by batch_count and img_count parameters.

after_epoch(epoch_id, **_)

Set _current_epoch_id which is used for distinguish between epochs directories. Call the _reset function.

property image_suffix

The suffix of the saved image, used to distinguish between images from different hooks.

Return type

str

process_img(img_i, batch_data)

This method is called in after_batch method and its purpose is to prepare image for save. If convenient, this method can be overridden in a subclass.

Return type

ndarray

to_color(image)

If the image is in grayscale, it will be converted to RGB.

class blocks.hooks.VisualizeMasks(output_dir, mask_variable='mask', mask_factor=255, mask_opacity=0.3, color=(0, 255, 0), **kwargs)

Bases: blocks.hooks.save_images.SaveImages

Join and save images with the corresponding segmentation masks after each batch.

Caution

Saving all the images may require considerable amount of time. Use this hook only for debugging or limit the number of saved images with image_count or batch_count parameters.

Join and save image stored in variable imgs with mask stored in variable segments.

hooks:
  - blocks.hooks.VisualizeMasks:
      variable: imgs
      mask_variable: mask

__init__(output_dir, mask_variable='mask', mask_factor=255, mask_opacity=0.3, color=(0, 255, 0), **kwargs)

Create new VisualizeMasks hook.

Parameters

• mask_variable (str) – name of the variable representing the mask of image

• mask_factor (float) – constant by which the mask is multiplied

• mask_opacity (float) – opacity of the mask used for composition with image

• color (Tuple[int, int, int]) – color of the mask used in the resulting image (BGR colorspace)

after_batch(stream_name, batch_data)

Assert mask variable is in batch data.

property image_suffix

Image suffix in the form: <variable>_<mask_variable>.

Return type

str

process_img(img_i, batch_data)

Join the image with dyed mask. Only the white pixels of the mask are dyed (which satisfies: mask_px * mask_factor = 255).

Return type

ndarray

class blocks.hooks.VisualizeMasksGT(output_dir, mask_variable='mask', predictions_variable='predictions', mask_opacity=0.8, mask_factor=255, predictions_factor=255, color_true=(0, 255, 0), color_false_pos=(0, 0, 255), color_false_neg=(255, 0, 0), **kwargs)

Bases: blocks.hooks.save_images.SaveImages

Join and save image with dyed segmentation mask prediction.

Caution

Saving all the images may require considerable amount of time. Use this hook only for debugging or limit the number of saved images with image_count or batch_count parameters.

Join image stored in variable imgs with dyed layer obtained from mask stored in variable segments and predictions stored in variable results.

hooks:
  - blocks.hooks.VisualizeMasksGT:
      variable: imgs
      mask_variable: mask
      predictions_variable: results

__init__(output_dir, mask_variable='mask', predictions_variable='predictions', mask_opacity=0.8, mask_factor=255, predictions_factor=255, color_true=(0, 255, 0), color_false_pos=(0, 0, 255), color_false_neg=(255, 0, 0), **kwargs)

Create new VisualizeMasksGT hook.

Parameters

• mask_variable (str) – name of the variable representing the mask of image

• predictions_variable (str) – name of the variable representing the predictions (model output)

• mask_opacity (float) – opacity of the mask used for composition with image

• mask_factor (float) – the multiplier of the mask

• predictions_factor (float) – the multiplier of the predictions variable

• color_true (Tuple[int, int, int]) – color for the true positive and true negative pixels (BGR colorspace)

• color_false_pos (Tuple[int, int, int]) – color for the false positive pixels (BGR colorspace)

• color_false_neg (Tuple[int, int, int]) – color for the false negative pixels (BGR colorspace)

after_batch(stream_name, batch_data)

Assert mask and prediction variables are in batch data.

property image_suffix

Image suffix in the form: <variable>_<mask_variable>_<predictions_variable>.

Return type

str

process_img(img_i, batch_data)

Join the image with the layer where pixels for true positive and true negative are dyed to color_true, pixels for false positive are dyed to color_false_pos and pixels for false negative are dyed to color_false_neg.

Return type

ndarray

class blocks.hooks.VisualizeRectangleProposals(output_dir, images_name='images', rectangles_name='rectangle_proposals', expected_rectangles_name='target_regions', errors_only=False, **kwargs)

Bases: hooks.AbstractHook

Visualize proposed rectangle regions.

__init__(output_dir, images_name='images', rectangles_name='rectangle_proposals', expected_rectangles_name='target_regions', errors_only=False, **kwargs)

Create new VisualizeRectangleProposals hook.

Parameters

• output_dir (str) – output directory

• images_name (str) – images variable name

• rectangles_name (str) – rectangles variable name

• expected_rectangles_name (str) – expected rectangles variable name

• errors_only (bool) – visualize only images with recall < 1.0

after_batch(stream_name, batch_data)

Visualize rectangle proposals and save the results to the output_dir.

Return type

None

class blocks.hooks.ComputeRegionProposals(dataset, non_maximum_suppression=0.3, min_probability=None, top_k=None, classifier_probabilities_name='classifier_probabilities', regression_predictions_name='regression_predictions', proposed_regions_name='rectangle_proposals', **kwargs)

Bases: hooks.AbstractHook

__init__(dataset, non_maximum_suppression=0.3, min_probability=None, top_k=None, classifier_probabilities_name='classifier_probabilities', regression_predictions_name='regression_predictions', proposed_regions_name='rectangle_proposals', **kwargs)

Create new ComputeRectangleProposal hook.

Parameters

• dataset (RectangleRPNDataset) – RPN dataset which created the anchors

• non_maximum_suppression (Optional[float]) – if specified, use nms with the given threshold

• min_probability (Optional[float]) – if specified, filter out regions with probability lower than the specified threshold

• top_k (Optional[int]) – if specified, output only top k regions

• classifier_probabilities_name (str) – name of the classifier probabilities variable

• regression_predictions_name (str) – name of the regression predictions variable

• proposed_regions_name (str) – name of the output proposed regions variable

after_batch(stream_name, batch_data)

Decode RPN predictions and save the proposed regions to batch data.

Return type

None

class blocks.hooks.ComputeRegionStats(dataset, expected_rectangles_name='target_regions', predicted_rectangles_name='rectangle_proposals', **kwargs)

Bases: hooks.AbstractHook

Compute rectangle RPN stats (recall and overlap ratio).

__init__(dataset, expected_rectangles_name='target_regions', predicted_rectangles_name='rectangle_proposals', **kwargs)

Create new ComputeRectangleRPNStats hook.

Parameters

• dataset (RectangleRPNDataset) – RectangleRPNDataset which created the anchors

• expected_rectangles_name (str) – expected rectangles variable name

• predicted_rectangles_name (str) – predicted rectangles variable name

after_batch(stream_name, batch_data)

Compute recall and overlap ratio for the expected and predicted rectangles.

Return type

None

class blocks.hooks.VisualizeThresholdedPrediction(output_dir, image_variable, probability_variable, n_batches=1, n_epochs=1, thresholds=None, mask_opacity=0.5, color=(0, 255, 0), **kwargs)

Bases: hooks.AbstractHook

Save images overlayed by their semi-transparent prediction masks for each of the given thresholds. The entire batch is saved as a single image matrix where columns correspond to given thresholds and rows to images in batch.

__init__(output_dir, image_variable, probability_variable, n_batches=1, n_epochs=1, thresholds=None, mask_opacity=0.5, color=(0, 255, 0), **kwargs)

Parameters

• output_dir (str) – output directory where masks will be saved

• image_variable (str) – name of the variable representing the source image

• probability_variable (str) – name of the variable representing the probability image

• n_batches (int) – count of batches from which the masks will be saved

• n_epochs (int) – count of epochs from which the masks will be saved

• thresholds (Optional[Iterable[float]]) – list of probability thresholds from which the masks will be saved

• mask_opacity (float) – opacity of the mask used for composition with image

• color (Tuple[int, int, int]) – color of the mask used for composition with image

after_batch(stream_name, batch_data)

Save the masks for each example.

after_epoch(epoch_id, **kwargs)

After epoch event.

This event is triggered after every epoch wherein all the streams were iterated. The epoch_data object is initially empty and shared among all the hooks.

Parameters

• epoch_id (int) – finished epoch id

• epoch_data – epoch data flowing through all hooks

class blocks.hooks.Threshold(input_variable, output_variable, threshold, **kwargs)

Bases: hooks.AbstractHook

Threshold image.

__init__(input_variable, output_variable, threshold, **kwargs)

Parameters

• input_variable (str) – name of the variable representing the probability image

• output_variable (str) – name of the variable to which threshold image will be saved

• threshold (float) – the values above this threshold will be set to 1 and the rest to 0

after_batch(stream_name, batch_data)

After batch event.

This event is triggered after every processed batch regardless of stream type. Batch results are available in results argument.

Parameters

• stream_name (str) – name of the stream (usually train, valid or``test``)

• batch_data (Mapping[str, Sequence[Any]]) – batch inputs and model outputs

class blocks.hooks.SaveNumpy(variable, id_variable, output_dir, **kwargs)

Bases: hooks.AbstractHook

Dump numpy arrays.

__init__(variable, id_variable, output_dir, **kwargs)

Parameters

• variable (str) – name of the variable representing the numpy array

• id_variable (str) – name of the variable which represents a unique example id

• output_dir (str) – output directory where masks will be saved

after_batch(stream_name, batch_data)

Save the masks for each example.

after_epoch(epoch_id, **kwargs)

Set _current_epoch_id which is used to distinguish between epoch directories. Reset _batch_count to initial value.