DerivedDataset#

class nominal.experimental.derived_datasets.DerivedDataset(rid: str, name: str, description: str | None, properties: NominalProperties, labels: Sequence[str], is_archived: bool)#

Bases: object

A dataset whose contents are computed from a definition instead of ingested files.

Returned by create_derived_dataset, and by get_derived_dataset for a dataset the server reports as derived. The *_input_dataset* methods read and edit a definition authored as a union of input datasets, each optionally filtered, tagged and time-shifted — the same composition the app builds. get_definition and commit_definition work with any definition as a raw compute spec.

It is not a Dataset: nothing can be ingested into it. To read its data or attach it to an asset or a run, look it up as a regular dataset with NominalClient.get_dataset by rid.

rid: str#
name: str#
description: str | None#
properties: Mapping[str, str | float]#
labels: Sequence[str]#
is_archived: bool#
add_input_dataset(dataset_input: DerivedDatasetInput, *, message: str | None = None) → Sequence[DerivedDatasetInput]#

Add an input dataset to this derived dataset, as a new commit on its definition.

Parameters:

  • dataset_input – The dataset to add, with its transforms. Build one with DerivedDatasetInput.create, which takes a Dataset or a RID.

  • message – Commit message describing the change. Defaults to naming the added dataset.

Returns:

The derived dataset’s input datasets after the change.

Raises:

  • ValueError – If this dataset’s definition was authored as something other than a combination of input datasets.

  • conjure_python_client.ConjureHTTPError – If the definition was committed to by someone else since this call read it.

archive() → None#

Archive this dataset. Archived datasets are not deleted, but are hidden from the UI.

commit_definition(spec: Dataset, message: str, *, latest_commit: str | None = None) → DerivedDefinition#

Replace this dataset’s derived definition with spec, as a new commit.

Parameters:

  • spec – The compute spec defining the new contents.

  • message – Commit message describing the change.

  • latest_commit – If provided, the commit this change was based on. The server refuses the write if the definition has moved on since.

Returns:

The newly committed derived definition.

get_definition(commit: str | None = None) → DerivedDefinition#

Fetch this dataset’s derived definition: its compute spec and the commit that produced it.

Parameters:

commit – If provided, the definition at this commit rather than the latest.

Returns:

The derived definition at the requested commit, or the latest if none was given.

list_input_datasets() → Sequence[DerivedDatasetInput]#

List the input datasets this derived dataset is composed of, with the transforms applied to each.

Returns:

One DerivedDatasetInput per appearance in the definition, in definition order.

Raises:

ValueError – If this dataset’s definition was authored as something other than a combination of input datasets with the per-input transforms this API models.

remove_input_dataset(dataset: Dataset | str | DerivedDatasetInput, *, message: str | None = None) → Sequence[DerivedDatasetInput]#

Remove an input dataset from this derived dataset, as a new commit on its definition.

Pass one of the inputs list_input_datasets returned to remove the appearances equal to it, which differ from the others by their transforms. Passing a dataset or a RID removes every appearance of it — a dataset added more than once, under different filters or at different offsets, is how a single source is aligned against itself.

Parameters:

  • dataset – The input to remove, or the dataset (or its RID) to remove every appearance of.

  • message – Commit message describing the change. Defaults to naming the removed dataset.

Returns:

The derived dataset’s input datasets after the change.

Raises:

  • ValueError – If nothing matching is an input of this one, or if this dataset’s definition was authored as something other than a combination of input datasets.

  • conjure_python_client.ConjureHTTPError – If the definition was committed to by someone else since this call read it.

set_input_datasets(inputs: Sequence[DerivedDatasetInput], *, message: str | None = None) → Sequence[DerivedDatasetInput]#

Replace this derived dataset’s input datasets wholesale, as a new commit on its definition.

Use this to change an input’s transforms, reorder the inputs, or make several changes in one commit. Start from list_input_datasets and pass back the edited sequence.

Parameters:

  • inputs – The inputs the definition should have after the change, in order. May be empty.

  • message – Commit message describing the change. Defaults to counting the inputs.

Returns:

The derived dataset’s input datasets after the change.

Raises:

  • ValueError – If this dataset’s definition was authored as something other than a combination of input datasets, which this method would otherwise overwrite.

  • conjure_python_client.ConjureHTTPError – If the definition was committed to by someone else since this call read it.