get_data_stream()#

relationalai.api
#get_data_stream(object_fq_name STRING, model_name STRING)

Retrieves information about a data stream for the specified RAI Python model, including when the stream was created, its status, and synchronization details. Requires the cdc_admin application role.

Parameters#

NameTypeDescription
object_fq_nameSTRINGThe fully-qualified name of the data stream’s source table or view in Snowflake, e.g. '<db>.<schema>.<table_or_view>'.
model_nameSTRINGThe name of the model associated with the stream.

Returns#

A table with the following schema:

Column NameData TypeDescription
IDSTRINGThe data stream’s unique identifier.
CREATED_ATTIMESTAMPThe timestamp when the data stream was created.
CREATED_BYSTRINGThe user who created the data stream.
STATUSSTRINGThe data stream’s current status. May be one of:
  • CREATED
  • DELETING
To get the stream’s current synchronization status, use the api.get_data_stream() procedure.
REFERENCE_NAMESTRINGThe type of the stream’s object reference. May be one of
  • DATA_STREAM_TABLE
  • DATA_STREAM_VIEW
REFERENCE_ALIASSTRINGThe unique identifier for the stream’s object reference.
FQ_OBJECT_NAMESTRINGThe fully-qualified name of the stream’s source table or view.
RAI_DATABASESTRINGThe name of the RAI Python model for which the stream was created.
RAI_RELATIONSTRINGThe name of the stream as passed to the stream_name parameter of the api.create_data_stream() procedure.
DATA_SYNC_STATUSSTRINGThe data stream’s synchronization status. May be one of:
  • SYNCED
  • SYNCING
  • QUARINTINED
  • SUSPENDED
See Quarantined Streams for details on what it means to be quarantined.
PENDING_BATCHES_COUNTINTThe number of pending batches left to process for the stream.
NEXT_BATCH_STATUSSTRINGThe status of the next batch to be processed.
NEXT_BATCH_UNLOADED_TIMESTAMPTIMESTAMPThe timestamp when the next batch will be unloaded.
NEXT_BATCH_DETAILSOBJECTA JSON object containing details about the next batch to be processed. Contains the same data as the api.data_stream_batches view.
LAST_BATCH_DETAILSOBJECTA JSON object containing details about the last batch processed. Contains the same data as the api.data_stream_batches view.
LAST_BATCH_UNLOADED_TIMESTAMPTIMESTAMPThe timestamp when the last batch was unloaded.
LAST_TRANSACTION_IDSTRINGThe transaction ID of the last batch process. Use the api.transactions view to get transaction details.
ERRORSARRAYAn array of error messages, if any, encountered during stream processing.
CDC_STATUSSTRINGThe status of the CDC service.

Example#

Use api.get_data_stream() to retrieve information about a data stream for a RAI Python model:

#-- Get information about a data stream in a model named 'MyModel'. Replace the
-- placeholders with your database, schema, and table or view name.
CALL relationalai.api.get_data_stream('<db>.<schema>.<table_or_view>', 'MyModel');
/*+-----------------------------------------+-------------------------+-----------------------+--------+-------------------+--------------------------------------+-------------------------------+--------------+-------------------------------+------------------+------------------------+-------------------+-------------------------------+---------------------------------+-------------------------------+-------------------------------+----------------------------------------+--------+------------+
  | ID                                      | CREATED_AT              | CREATED_BY            | STATUS | REFERENCE_NAME    | REFERENCE_ALIAS                      | FQ_OBJECT_NAME                | RAI_DATABASE | RAI_RELATION                  | DATA_SYNC_STATUS | PENDING_BATCHES_COUNT  | NEXT_BATCH_STATUS | NEXT_BATCH_UNLOADED_TIMESTAMP | NEXT_BATCH_DETAILS              | LAST_BATCH_DETAILS            | LAST_BATCH_UNLOADED_TIMESTAMP | LAST_TRANSACTION_ID                    | ERRORS | CDC_STATUS |
  |-----------------------------------------+-------------------------+-----------------------+--------+-------------------+--------------------------------------+-------------------------------+--------------+-------------------------------+------------------+------------------------+-------------------+-------------------------------+---------------------------------+-------------------------------+-------------------------------+----------------------------------------+--------+------------|
  | ds_abcd1234_ef56_7890_abcd_1234ef567890 | 2024-10-23 10:12:34.567 | jane.doe@example.com  | ACTIVE | DATA_STREAM_TABLE | a1bcdef2-3456-7890-1234-b567c890d123 | <db>.<schema>.<table_or_view> | MyModel      | <db>.<schema>.<table_or_view> | SYNCED           | 0                      | NULL              | NULL                          | NULL                            | {"rows": 10, "size": 512, ... } 2024-10-23 10:50:00.456       | 02a1b234-5678-1234-abcdef-0123456789ab | []     | STARTED    |
  +-----------------------------------------+-------------------------+-----------------------+--------+-------------------+--------------------------------------+-------------------------------+--------------+-------------------------------+------------------+------------------------+-------------------+-------------------------------+---------------------------------+-------------------------------+-------------------------------+----------------------------------------+--------+------------+ */

See Data Management for more information data streams.

See Also#