data_streams#

relationalai.api
#data_streams

View that lists all data streams created in the RAI Native App. Provides information about each stream, including when it was created and the name of the source table or view in Snowflake. To get the stream’s current synchronization status, use the api.get_data_stream() procedure. Requires the cdc_admin application role.

Columns#

Column NameData TypeDescription
IDSTRINGThe data stream’s unique identifier.
CREATED_ATTIMESTAMPThe timestamp when the data stream was created.
CREATED_BYSTRINGThe user who created the data stream.
STATUSSTRINGThe data stream’s current status. May be one of:
  • CREATED
  • DELETING
  • CANNOT_READ_SOURCE
For details on each status, see Data Stream Statuses. To get the stream’s current synchronization status, use the api.get_data_stream() procedure and refer to the DATA_SYNC_STATUS column.
REFERENCE_NAMESTRINGThe type of the stream’s object reference. May be one of
  • DATA_STREAM_TABLE
  • DATA_STREAM_VIEW
REFERENCE_ALIASSTRINGThe unique identifier for the stream’s object reference.
FQ_OBJECT_NAMESTRINGThe fully-qualified name of the stream’s source table or view.
RAI_DATABASESTRINGThe name of the RAI Python model for which the stream was created.
RAI_RELATIONSTRINGThe name of the stream as passed to the stream_name parameter of the api.create_data_stream() procedure. Typically the same as FQ_OBJECT_NAME.
VERSIONSTRINGThe version number of the data stream.

Data Stream Statuses#

StatusDescription
CREATEDThe stream has been created. Note that this does not indicate that the stream is active and syncing. To get the stream’s current data synchronization status, call the api.get_data_streams() procedure and refer to the DATA_SYNC_STATUS column.
DELETINGThe stream has been marked for deletion and will be removed.
CANNOT_READ_SOURCEThe stream cannot read the source table or view. This happens when the stream’s reference to the source table or view in Snowflake has been removed, for example, by calling the SYSTEM$REMOVE_REFERENCE Snowflake SQL function. The data stream must be deleted and then recreated.

Example#

Use the api.data_streams view to query information about data streams:

#SELECT * FROM relationalai.api.data_streams;
/*+-------------------------------------+-------------------------+--------------------------+----------+-------------------+--------------------------------------+------------------------- -+--------------+---------------------------+ --------+
  | ID                                  | CREATED_AT              | CREATED_BY               | STATUS   | REFERENCE_NAME    | REFERENCE_ALIAS                      | FQ_OBJECT_NAME            | RAI_DATABASE | RAI_RELATION              | VERSION |
  |-------------------------------------+-------------------------+--------------------------+----------+-------------------+--------------------------------------+---------------------------+--------------+---------------------------+---------|
  | ds_a1b2c3d4_e5f6_7a89_b123_d456e789 | 2024-10-23 12:23:45.250 | john.doe@company.com     | CREATED  | DATA_STREAM_VIEW  | 1234abcd-5678-90ef-ab12-3456cdef7890 | example_db.sales.view1    | SalesModel   | example_db.sales.view1    | 0.0     |
  | ds_8e7f6d5c_4a3b_2c1d_0e9f_7b6a8d9f | 2024-10-22 15:37:29.580 | maria.garcia@company.com | CREATED  | DATA_STREAM_TABLE | bcd123ef-4567-890a-bcde-abcdef678901 | example_db.hr.employees   | HRModel      | example_db.hr.employees   | 0.0     |
  | ds_9a8b7c6d_5e4f_3d2a_1b0e_f7g6h5i3 | 2024-10-21 17:44:10.300 | mark.jones@company.com   | DELETING | DATA_STREAM_VIEW  | 7890abcd-1234-5678-90ef-bcde4567890f | example_db.finance.budget | FinanceModel | example_db.finance.budget | 0.0     |
  +-------------------------------------+-------------------------+--------------------------+----------+-------------------+--------------------------------------+---------------------------+--------------+---------------------------+---------+ */

For instance, you can filter the results to show only streams created by a specific user:

#SELECT * FROM relationalai.api.data_streams WHERE CREATED_BY = 'john.doe@company.com';
/*+-------------------------------------+-------------------------+--------------------------+----------+-------------------+--------------------------------------+------------------------- -+--------------+--------------------------+
  | ID                                  | CREATED_AT              | CREATED_BY               | STATUS   | REFERENCE_NAME    | REFERENCE_ALIAS                      | FQ_OBJECT_NAME            | RAI_DATABASE | RAI_RELATION             |
  |-------------------------------------+-------------------------+--------------------------+----------+-------------------+--------------------------------------+---------------------------+--------------+--------------------------|
  | ds_a1b2c3d4_e5f6_7a89_b123_d456e789 | 2024-10-23 12:23:45.250 | john.doe@company.com     | CREATED  | DATA_STREAM_VIEW  | 1234abcd-5678-90ef-ab12-3456cdef7890 | example_db.sales.view1    | SalesModel   | example_db.sales.view1   |
  +-------------------------------------+-------------------------+--------------------------+----------+-------------------+--------------------------------------+---------------------------+--------------+--------------------------+ */

Use the api.get_data_stream() procedure to get more detailed information about a specific stream, such as its synchronization status. See Data Management for more information about data streams.

See Also#