DerivedDataset#

class nominal.experimental.derived_datasets.DerivedDataset(rid, name, description, properties, labels, is_archived)#

Bases: object

A dataset whose contents are computed from a definition instead of ingested files.

Returned by create_derived_dataset, and by get_derived_dataset for a dataset the server reports as derived. The *_input_dataset* methods read and edit a definition authored as a union of input datasets, each optionally filtered, tagged and time-shifted — the same composition the app builds. get_definition and commit_definition work with any definition as a raw compute spec.

It is not a Dataset: nothing can be ingested into it. To read its data or attach it to an asset or a run, look it up as a regular dataset with NominalClient.get_dataset by rid.

rid: str#
name: str#
description: str | None#
properties: Mapping[str, str | float]#
labels: Sequence[str]#
is_archived: bool#
add_input_dataset(dataset_input, *, message=None)#

Add an input dataset to this derived dataset, as a new commit on its definition.

Parameters:

  • dataset_input (DerivedDatasetInput)

    The dataset to add, with its transforms. Build one with DerivedDatasetInput.create, which takes a Dataset or a RID.

  • message (str | None)

    Commit message describing the change. Defaults to naming the added dataset.

Returns:

The derived dataset’s input datasets after the change.

Raises:

  • ValueError – If this dataset’s definition was authored as something other than a combination of input datasets.

  • conjure_python_client.ConjureHTTPError – If the definition was committed to by someone else since this call read it.

Return type:

Sequence[DerivedDatasetInput]

archive()#

Archive this dataset. Archived datasets are not deleted, but are hidden from the UI.

commit_definition(spec, message, *, latest_commit=None)#

Replace this dataset’s derived definition with spec, as a new commit.

Parameters:

  • spec (Dataset)

    The compute spec defining the new contents.

  • message (str)

    Commit message describing the change.

  • latest_commit (str | None)

    If provided, the commit this change was based on. The server refuses the write if the definition has moved on since.

Returns:

The newly committed derived definition.

Return type:

DerivedDefinition

get_definition(commit=None)#

Fetch this dataset’s derived definition: its compute spec and the commit that produced it.

Parameters:

commit (str | None)

If provided, the definition at this commit rather than the latest.

Returns:

The derived definition at the requested commit, or the latest if none was given.

Return type:

DerivedDefinition

list_input_datasets()#

List the input datasets this derived dataset is composed of, with the transforms applied to each.

Returns:

One DerivedDatasetInput per appearance in the definition, in definition order.

Raises:

ValueError – If this dataset’s definition was authored as something other than a combination of input datasets with the per-input transforms this API models.

Return type:

Sequence[DerivedDatasetInput]

remove_input_dataset(dataset, *, message=None)#

Remove an input dataset from this derived dataset, as a new commit on its definition.

Pass one of the inputs list_input_datasets returned to remove the appearances equal to it, which differ from the others by their transforms. Passing a dataset or a RID removes every appearance of it — a dataset added more than once, under different filters or at different offsets, is how a single source is aligned against itself.

Parameters:

  • dataset (Dataset | str | DerivedDatasetInput)

    The input to remove, or the dataset (or its RID) to remove every appearance of.

  • message (str | None)

    Commit message describing the change. Defaults to naming the removed dataset.

Returns:

The derived dataset’s input datasets after the change.

Raises:

  • ValueError – If nothing matching is an input of this one, or if this dataset’s definition was authored as something other than a combination of input datasets.

  • conjure_python_client.ConjureHTTPError – If the definition was committed to by someone else since this call read it.

Return type:

Sequence[DerivedDatasetInput]

set_input_datasets(inputs, *, message=None)#

Replace this derived dataset’s input datasets wholesale, as a new commit on its definition.

Use this to change an input’s transforms, reorder the inputs, or make several changes in one commit. Start from list_input_datasets and pass back the edited sequence.

Parameters:

  • inputs (Sequence[DerivedDatasetInput])

    The inputs the definition should have after the change, in order. May be empty.

  • message (str | None)

    Commit message describing the change. Defaults to counting the inputs.

Returns:

The derived dataset’s input datasets after the change.

Raises:

  • ValueError – If this dataset’s definition was authored as something other than a combination of input datasets, which this method would otherwise overwrite.

  • conjure_python_client.ConjureHTTPError – If the definition was committed to by someone else since this call read it.

Return type:

Sequence[DerivedDatasetInput]