cls_luigi.cls_tasks

This file contains the CLSBaseTask implementation, which is used to extend the actual implementation of luigi.Task, luigi.WrapperTask and luigi.ExternalTask with helper methods to make the usage of our framework a bit easier.

Module Contents

Classes

ClsBaseTask

This class is used to implement some helper methods that all tasks in a CLS-Luigi pipeline should have.

ClsTask

Abstract class representing a CLS-Luigi task. It combines the functionality of luigi.Task, LuigiCombinator, and ClsBaseTask.

ClsWrapperTask

Abstract class representing a CLS-Luigi wrapper task. It combines the functionality of luigi.WrapperTask, LuigiCombinator, and ClsBaseTask.

ClsExternalTask

Abstract class representing a CLS-Luigi external task. It combines the functionality of luigi.ExternalTask, LuigiCombinator, and ClsBaseTask.
class cls_luigi.cls_tasks.ClsBaseTask[source]

This class is used to implement some helper methods that all tasks in a CLS-Luigi pipeline should have.

The key methods are:

  • get_variant_filename() - This method creates a unique filename for (intermediate) results, which remains unique across the executed pipelines. For this purpose a hash value is created and persisted to disk, which can be looked up afterwards to trace the executed tasks back to this file.

initialized = False[source]
__get_flatten_data(data: luigi.target.FileSystemTarget | tuple | dict) → list[source]

Helper method to flatten different possible input data types into a flattened list.

Parameters:

data (luigi.target.FileSystemTarget | tuple | dict) – the data structure that should be flattened to a list.

Returns:

a flattened list of the data provided.

Return type:

list

get_variant_filename(name='') → str[source]

Returns a variant filename based on the provided name. Also does hashing of the name, since is has to be unique over every pipeline.

Parameters:

name (str, optional) – optional name for the variant, defaults to “”

Returns:

the variant filename.

Return type:

str

__helper_variant_label(input_file)[source]
create_result_file(file_name)[source]
class cls_luigi.cls_tasks.ClsTask(*args, **kwargs)[source]

Bases: luigi.Task, cls_luigi.inhabitation_task.LuigiCombinator, ClsBaseTask

Abstract class representing a CLS-Luigi task. It combines the functionality of luigi.Task, LuigiCombinator, and ClsBaseTask.

abstract = True[source]
class cls_luigi.cls_tasks.ClsWrapperTask(*args, **kwargs)[source]

Bases: luigi.WrapperTask, cls_luigi.inhabitation_task.LuigiCombinator, ClsBaseTask

Abstract class representing a CLS-Luigi wrapper task. It combines the functionality of luigi.WrapperTask, LuigiCombinator, and ClsBaseTask.

abstract = True[source]
class cls_luigi.cls_tasks.ClsExternalTask(*args, **kwargs)[source]

Bases: luigi.ExternalTask, cls_luigi.inhabitation_task.LuigiCombinator, ClsBaseTask

Abstract class representing a CLS-Luigi external task. It combines the functionality of luigi.ExternalTask, LuigiCombinator, and ClsBaseTask.

abstract = True[source]