as_files_ingested

nominal.core.as_files_ingested(files: Sequence[DatasetFile], *, poll_interval: timedelta = datetime.timedelta(seconds=1)) → Iterable[DatasetFile]

Iterates over DatasetFiles as they complete their ingestion in a similar fashion to concurrent.futures.as_completed.

Any files that are already ingested (successfully or with errors) will immediately be yielded.

Parameters:

• files – Dataset files to monitor for ingestion completion.

• poll_interval – Interval to sleep between polling the remaining files under watch.

Yields:

Yields DatasetFiles as they are ingested. Due to the polling mechanics, the files are not yielded in strictly sorted order based on their ingestion completion time. Ensure to check the ingest_status of yielded dataset files if important.