datasource_to_dataframe#

nominal.thirdparty.pandas.datasource_to_dataframe(datasource, channel_exact_match=None, channel_fuzzy_search_text=None, start=None, end=None, tags=None, enable_gzip=True, *, channels=None, num_workers=1, channel_batch_size=20, relative_to=None, relative_resolution='nanoseconds')#

Download a dataset to a pandas dataframe, optionally filtering for only specific channels of the dataset.

Parameters:

  • datasource (Dataset)

    The datasource to download data from

  • channel_exact_match (Sequence[str] | None)

    Filter the returned channels to those whose names match all provided strings (case insensitive). For example, a channel named ‘engine_turbine_rpm’ would match against [‘engine’, ‘turbine’, ‘rpm’], whereas a channel named ‘engine_turbine_flowrate’ would not!

  • channel_fuzzy_search_text (str | None)

    Filters the returned channels to those whose names fuzzily match the provided string.

  • channels (Sequence[Channel] | None)

    List of channels to fetch data for. If provided, supercedes search parameters of channel_exact_match and channel_fuzzy_search_text.

  • tags (Mapping[str, str] | None)

    Dictionary of tags to filter channels by

  • start (str | datetime | int | None)

    The minimum data updated time to filter channels by

  • end (str | datetime | int | None)

    The maximum data start time to filter channels by

  • enable_gzip (bool)

    If true, use gzip when exporting data from Nominal. This will almost always make export faster and use less bandwidth.

  • num_workers (int)

    Use this many parallel processes for performing export requests against the backend. This should roughly be corresponding to the strength of your network connection, with 4-8 workers being more than sufficient to completely saturate most connections.

  • channel_batch_size (int)

    Number of channels to request at a time per worker thread. Reducing this number may allow fetching a larger time duration (i.e., end - start), depending on how synchronized the timing is amongst the requested channels. This is a result of a limit of 10_000_000 unique timestamps returned per request, so reducing the number of channels will allow for a larger time window if channels come in at different times (e.g. channel A has timestamps 100, 200, 300… and channel B has timestamps 101, 201, 301, …). This is particularly useful when combined with num_workers when attempting to maximally utilize a machine.

  • relative_to (datetime | int | None)

    If provided, return timestamps relative to the given epoch time

  • relative_resolution (Literal['picoseconds', 'nanoseconds', 'microseconds', 'milliseconds', 'seconds', 'minutes', 'hours', 'days'])

    If providing timestamps in relative time, the resolution to use

Returns:

A pandas dataframe whose index is the timestamp of the data, and column names match those of the selected

channels.

Return type:

DataFrame

Example

rid = "..." # Taken from the UI or via the SDK
dataset = client.get_dataset(rid)
df = datasource_to_dataframe(dataset)
print(df.head())  # Show first few rows of data