RelationalAI Engine Management#
RelationalAI (RAI) engines are isolated compute units similar to Snowflake virtual warehouses. They execute queries on RAI schemas and process change data capture (CDC) streams. Like Snowflake warehouses, engines can be independently created for workload isolation or scaling.
Managing Engines#
You can manage engines using the RAI CLI
or SQL procedures provided by the RAI Native App.
To manage engines, you must have eng_admin
application role privileges.
Create an Engine#
Use the create_engine()
procedure to create an engine:
#CALL relationalai.api.create_engine('<engine_name>', '<engine_size>');
The engine size must be one of the following:
'HIGHMEM_X64_S'
for small engines'HIGHMEM_X64_M'
for medium engines
Engines run the models and queries written by users of the relationalai
Python package.
Small engines are recommended as the default for most workloads.
Medium engines are recommended for models and queries involving large-scale graph algorithms and complex computations.
See Performance Considerations for tips on monitoring engine transactions and determining the appropriate engine size.
By default, engines can’t access endpoints outside of your Snowflake account. Refer to the reference docs for details on enabling external integrations.
Get Engine Details#
Use the get_engine()
procedure to get details about an engine:
#-- NOTE: Engine names are case-sensitive.
CALL relationalai.api.get_engine('<engine_name>');
Delete an Engine#
To delete an engine, use the delete_engine()
procedure:
#-- NOTE: Engine names are case-sensitive.
CALL relationalai.api.delete_engine('<engine_name>');
You cannot delete an engine that is currently used to process CDC streams.
To delete a CDC engine, use the setup_cdc()
procedure to reassign the stream to another engine
before deleting the original engine.
List All Engines#
To view a list of all engines you have access to, query the engines
view:
#SELECT * FROM relationalai.api.engines;
Performance Considerations#
This section provides an overview of performance considerations and outlines best practices for managing engine workloads effectively.
Managing Concurrent Workloads#
Engines provide performance isolation, but load balancing is manual. When multiple workloads run concurrently on the same engine, they may affect each other’s performance. To avoid coordination issues, use separate engines for different workloads.
Each engine can handle up to 8 queries concurrently and has a queue capacity of up to 128 transactions, with a first-in/first-out priority. However, resource-intensive queries execute sequentially.
Monitoring Queries#
Use the RAI CLI’s transactions:list
command to monitor queries:
#rai transactions:list
Take note of the following statuses. While occasional occurrences are normal, a high volume may indicate potential issues:
QUEUED
: The query is in the engine’s queue. If it remains in this state, the engine’s concurrency limit has been reached. Wait for the query to leave the queue or cancel it and rerun on an idle engine.CREATED
: The query is accepted but not yet in the engine queue. If it remains in this state, no engines are available, or resources are at capacity. Consider increasing engine size or adding more engines.
Best Practices#
- Avoid running multiple large-scale graph algorithms concurrently on the same engine.
- Limit interactions with multiple RAI schemas via a single engine to maintain optimal performance.
- Delete engines when not in use to reduce costs, but be mindful of warm-up costs for frequently restarted engines.
- Keep engines managing CDC streams active to ensure immediate data availability.