9.2.6. Pipeline¶

Pipeline components.

class junifer.pipeline.MarkerCollection(markers, datareader=None, preprocessors=None, storage=None)¶

Class for marker collection.

Parameters:

markerslist of marker-like: The markers to compute.
datareaderDataReader-like object, optional: The DataReader to use (default None).
preprocessorslist of preprocessing-like, optional: The preprocessors to apply (default None).
storagestorage-like, optional: The storage to use (default None).

Raises:

ValueError: If markers have same names.

fit(input)¶

Fit the pipeline.

Parameters:

inputdict: The input data to fit the pipeline on. Should be the output of indexing the Data Grabber with one element.

Returns:

dict or None: The output of the pipeline. Each key represents a marker name and the values are the computer marker values. If the pipeline has a storage configured, then the output will be None.

validate(datagrabber)¶

Validate the pipeline.

Without doing any computation, check if the marker collection can be fitted without problems i.e., the data required for each marker is present and streamed down the steps. Also, if a storage is configured, check that the storage can handle the markers’ output.

Parameters:

datagrabberDataGrabber-like: The DataGrabber to validate.

class junifer.pipeline.PipelineStepMixin¶

Mixin class for a pipeline step.

fit_transform(input, **kwargs)¶

Fit and transform.

Parameters:

inputdict: The Junifer Data object.
**kwargsdict: Extra keyword arguments passed to the concrete class’ _fit_transform().

Returns:

dict: The processed output of the pipeline step.

get_output_type(input_type)¶

Get output type.

Parameters:

input_typestr: The data type input to the marker.

Returns:

str: The storage type output by the marker.

validate(input)¶

Validate the the pipeline step.

Parameters:

inputlist of str: The input to the pipeline step.

Returns:

list of str: The output of the pipeline step.

validate_input(input)¶

Validate the input to the pipeline step.

Parameters:

inputlist of str: The input to the pipeline step. The list must contain the available Junifer Data dictionary keys.

Returns:

list of str: The actual elements of the input that will be processed by this pipeline step.

Raises:

ValueError: If the input does not have the required data.

class junifer.pipeline.UpdateMetaMixin¶

Mixin class for updating meta.

update_meta(input, step_name)¶

Update metadata.

Parameters:

inputdict: The data object to update.
step_namestr: The name of the pipeline step.

junifer.pipeline.WorkDirManager(*args, **kwargs)¶

Get the only instance for a class.

Parameters:

*argstuple: The positional arguments to pass to the class.
**kwargsdict: The keyword arguments to pass to the class.

Returns:

class: The only instance of the class.

9.2.7. Registry¶

Provide functions for registry.

junifer.pipeline.registry.build(step, name, baseclass, init_params=None)¶

Ensure that the given object is an instance of the given class.

Parameters:

stepstr: Name of the step.
namestr: Name of the function.
baseclassclass: Class to be checked against.
init_paramsdict or None, optional: Parameters to pass to the base class constructor (default None).

Returns:

object: An instance of the given base class.

Raises:

RuntimeError: If there is a problem creating the instance.
ValueError: If the created object with the given name is not an instance of the base class.

junifer.pipeline.registry.get_class(step, name)¶

Get the class of the registered function for a given step.

Parameters:

stepstr: Name of the step.
namestr: Name of the function.

Returns:

class: Registered function class.

Raises:

ValueError: If the step or name is invalid.

junifer.pipeline.registry.get_step_names(step)¶

Get the names of the registered functions for a given step.

Parameters:

stepstr: Name of the step.

Returns:

list: List of registered function names.

Raises:

ValueError: If the step is invalid.

junifer.pipeline.registry.register(step, name, klass)¶

Parameters:

stepstr: Name of the step.
namestr: Name of the function.
klassclass: Class to be registered.

Raises:

ValueError: If the step is invalid.