get_data_stream()#
relationalai.api
#get_data_stream(object_fq_name STRING, model_name STRING)
Retrieves information about a data stream for the specified RAI Python model, including when the stream was created, its status, and synchronization details.
Requires the cdc_admin
application role.
Parameters#
Name | Type | Description |
---|---|---|
object_fq_name | STRING | The fully-qualified name of the data stream’s source table or view in Snowflake, e.g. '<db>.<schema>.<table_or_view>' . |
model_name | STRING | The name of the model associated with the stream. |
Returns#
A table with the following schema:
Column Name | Data Type | Description |
---|---|---|
ID | STRING | The data stream’s unique identifier. |
CREATED_AT | TIMESTAMP | The timestamp when the data stream was created. |
CREATED_BY | STRING | The user who created the data stream. |
STATUS | STRING | The data stream’s current status. May be one of:
api.get_data_stream() procedure. |
REFERENCE_NAME | STRING | The type of the stream’s object reference. May be one of
|
REFERENCE_ALIAS | STRING | The unique identifier for the stream’s object reference. |
FQ_OBJECT_NAME | STRING | The fully-qualified name of the stream’s source table or view. |
RAI_DATABASE | STRING | The name of the RAI Python model for which the stream was created. |
RAI_RELATION | STRING | The name of the stream as passed to the stream_name parameter of the api.create_data_stream() procedure. |
DATA_SYNC_STATUS | STRING | The data stream’s synchronization status. May be one of:
|
PENDING_BATCHES_COUNT | INT | The number of pending batches left to process for the stream. |
NEXT_BATCH_STATUS | STRING | The status of the next batch to be processed. |
NEXT_BATCH_UNLOADED_TIMESTAMP | TIMESTAMP | The timestamp when the next batch will be unloaded. |
NEXT_BATCH_DETAILS | OBJECT | A JSON object containing details about the next batch to be processed. Contains the same data as the api.data_stream_batches view. |
LAST_BATCH_DETAILS | OBJECT | A JSON object containing details about the last batch processed. Contains the same data as the api.data_stream_batches view. |
LAST_BATCH_UNLOADED_TIMESTAMP | TIMESTAMP | The timestamp when the last batch was unloaded. |
LAST_TRANSACTION_ID | STRING | The transaction ID of the last batch process. Use the api.transactions view to get transaction details. |
ERRORS | ARRAY | An array of error messages, if any, encountered during stream processing. |
CDC_STATUS | STRING | The status of the CDC service. |
Example#
Use api.get_data_stream()
to retrieve information about a data stream for a RAI Python model:
#-- Get information about a data stream in a model named 'MyModel'. Replace the
-- placeholders with your database, schema, and table or view name.
CALL relationalai.api.get_data_stream('<db>.<schema>.<table_or_view>', 'MyModel');
/*+-----------------------------------------+-------------------------+-----------------------+--------+-------------------+--------------------------------------+-------------------------------+--------------+-------------------------------+------------------+------------------------+-------------------+-------------------------------+---------------------------------+-------------------------------+-------------------------------+----------------------------------------+--------+------------+
| ID | CREATED_AT | CREATED_BY | STATUS | REFERENCE_NAME | REFERENCE_ALIAS | FQ_OBJECT_NAME | RAI_DATABASE | RAI_RELATION | DATA_SYNC_STATUS | PENDING_BATCHES_COUNT | NEXT_BATCH_STATUS | NEXT_BATCH_UNLOADED_TIMESTAMP | NEXT_BATCH_DETAILS | LAST_BATCH_DETAILS | LAST_BATCH_UNLOADED_TIMESTAMP | LAST_TRANSACTION_ID | ERRORS | CDC_STATUS |
|-----------------------------------------+-------------------------+-----------------------+--------+-------------------+--------------------------------------+-------------------------------+--------------+-------------------------------+------------------+------------------------+-------------------+-------------------------------+---------------------------------+-------------------------------+-------------------------------+----------------------------------------+--------+------------|
| ds_abcd1234_ef56_7890_abcd_1234ef567890 | 2024-10-23 10:12:34.567 | jane.doe@example.com | ACTIVE | DATA_STREAM_TABLE | a1bcdef2-3456-7890-1234-b567c890d123 | <db>.<schema>.<table_or_view> | MyModel | <db>.<schema>.<table_or_view> | SYNCED | 0 | NULL | NULL | NULL | {"rows": 10, "size": 512, ... } 2024-10-23 10:50:00.456 | 02a1b234-5678-1234-abcdef-0123456789ab | [] | STARTED |
+-----------------------------------------+-------------------------+-----------------------+--------+-------------------+--------------------------------------+-------------------------------+--------------+-------------------------------+------------------+------------------------+-------------------+-------------------------------+---------------------------------+-------------------------------+-------------------------------+----------------------------------------+--------+------------+ */
See Data Management for more information data streams.