dimcat.steps.pipelines package#

Submodules#

dimcat.steps.pipelines.base module#

class dimcat.steps.pipelines.base.Pipeline(steps: Optional[Union[PipelineStep, Type[PipelineStep], DimcatConfig, dict, ObjectEnum, str, Iterable[Union[PipelineStep, Type[PipelineStep], DimcatConfig, dict, ObjectEnum, str]]]] = None, **kwargs)[source]#

Bases: PipelineStep

class Schema(*, only: Optional[Union[Sequence[str], AbstractSet[str]]] = None, exclude: Union[Sequence[str], AbstractSet[str]] = (), many: Optional[bool] = None, load_only: Union[Sequence[str], AbstractSet[str]] = (), dump_only: Union[Sequence[str], AbstractSet[str]] = (), partial: Optional[Union[bool, Sequence[str], AbstractSet[str]]] = None, unknown: Optional[Literal['exclude', 'include', 'raise']] = None)[source]#

Bases: Schema

dump_fields: dict[str, Field]#
exclude: set[Any] | MutableSet[Any]#
fields: dict[str, Field]#

Dictionary mapping field_names -> Field objects

load_fields: dict[str, Field]#
opts: Any = <marshmallow.schema.SchemaOpts object>#
unknown: types.UnknownOption#
add_step(step: Union[PipelineStep, Type[PipelineStep], DimcatConfig, dict, ObjectEnum, str]) → None[source]#
classmethod from_pipeline(pipeline: Pipeline) → Pipeline[source]#
classmethod from_step_configs(configs: Iterable[DimcatConfig]) → Pipeline[source]#
get_last_step(step_specs: Optional[Union[PipelineStep, Type[PipelineStep], DimcatConfig, dict, ObjectEnum, str]] = None, allow_subclasses: bool = True) → PipelineStep[source]#

Returns the last step that matches the given specs.

Parameters:

  • step_specs – Specification that can be converted to a DimcatConfig describing a PipelineStep. If None, the last step is returned.

  • allow_subclasses – By default, matches the last applied PipelineStep of the type described by step_specs or one of its subclasses. Set to False to return the last step that matches exactly.

Returns:

PipelineStep object that matches the given specs.

Raises:

NoMatchingPipelineStepFoundError – If no matching step is found.

get_steps(step_specs: Optional[Union[PipelineStep, Type[PipelineStep], DimcatConfig, dict, ObjectEnum, str]] = None, allow_subclasses: bool = True) → List[PipelineStep][source]#

Returns all steps that match the given specs.

Parameters:

  • step_specs – Specification that can be converted to a DimcatConfig describing a PipelineStep. If None, all steps are returned (equivalent to steps).

  • allow_subclasses – By default, matching subclasses of the PipelineStep described by step_specs are also included. Set to False to only return steps that match exactly.

Returns:

PipelineStep objects that matches the given specs.

info(return_str: Literal[False] = False) → None[source]#
info(return_str: Literal[True]) → str

Show the names of the included steps.

property steps: List[PipelineStep]#

The pipeline steps as a list. Modifying the list does not affect the pipeline.

Module contents#