datasource_to_dataframe#

nominal.thirdparty.pandas.datasource_to_dataframe(datasource: Dataset, channel_exact_match: Sequence[str] | None = None, channel_fuzzy_search_text: str | None = None, start: str | datetime | int | None = None, end: str | datetime | int | None = None, tags: Mapping[str, str] | None = None, enable_gzip: bool = True, *, channels: Sequence[Channel] | None = None, num_workers: int = 1, channel_batch_size: int = 20, relative_to: datetime | int | None = None, relative_resolution: Literal['picoseconds', 'nanoseconds', 'microseconds', 'milliseconds', 'seconds', 'minutes', 'hours', 'days'] = 'nanoseconds') → DataFrame#

Download a dataset to a pandas dataframe, optionally filtering for only specific channels of the dataset.

Parameters:

  • datasource – The datasource to download data from

  • channel_exact_match – Filter the returned channels to those whose names match all provided strings (case insensitive). For example, a channel named ‘engine_turbine_rpm’ would match against [‘engine’, ‘turbine’, ‘rpm’], whereas a channel named ‘engine_turbine_flowrate’ would not!

  • channel_fuzzy_search_text – Filters the returned channels to those whose names fuzzily match the provided string.

  • channels – List of channels to fetch data for. If provided, supercedes search parameters of channel_exact_match and channel_fuzzy_search_text.

  • tags – Dictionary of tags to filter channels by

  • start – The minimum data updated time to filter channels by

  • end – The maximum data start time to filter channels by

  • enable_gzip – If true, use gzip when exporting data from Nominal. This will almost always make export faster and use less bandwidth.

  • num_workers – Use this many parallel processes for performing export requests against the backend. This should roughly be corresponding to the strength of your network connection, with 4-8 workers being more than sufficient to completely saturate most connections.

  • channel_batch_size – Number of channels to request at a time per worker thread. Reducing this number may allow fetching a larger time duration (i.e., end - start), depending on how synchronized the timing is amongst the requested channels. This is a result of a limit of 10_000_000 unique timestamps returned per request, so reducing the number of channels will allow for a larger time window if channels come in at different times (e.g. channel A has timestamps 100, 200, 300… and channel B has timestamps 101, 201, 301, …). This is particularly useful when combined with num_workers when attempting to maximally utilize a machine.

  • relative_to – If provided, return timestamps relative to the given epoch time

  • relative_resolution – If providing timestamps in relative time, the resolution to use

Returns:

A pandas dataframe whose index is the timestamp of the data, and column names match those of the selected

channels.

Example

rid = "..." # Taken from the UI or via the SDK
dataset = client.get_dataset(rid)
df = datasource_to_dataframe(dataset)
print(df.head())  # Show first few rows of data