The unreasonable (cost) effectiveness of cloud-native GenAI

We recently encountered the simple problem of Entity Linking on contacts from different sources. The records have first names, last names, emails, and companies. The problem can be solved with simple algorithms. Imagine these two records that need to eventually be linked.

Record 1

First Name: Nikolaos
Last Name: Vasiloglou
Email: vasiloglou@gmail.com
Company: [empty]

Record 2

First Name: Nik
Last Name: Vasiloglou
Email: nik.vasiloglou@relational.ai
Company: RelationalAI

If you decide to solve the problem by using your server's computational resources, whether on-premise or hosted on the cloud, you would prefer to use a small machine to reduce the costs and focus on simple algorithms like rules, regex, and similarities such as edit distance. This can quickly become hairy and difficult to maintain. Christopher Re has described this as a wicked problem that looks easy but can cause death by a thousand cuts. Another method to solve this problem would be to use GenAI. But using GenAI sounds overkill because you would need an expensive GPU to run embeddings and an LLM at least 3B parameters big. Additionally, there's the operational burden of managing server uptime.

The approach shifts, however, if you can use embedding and LLM completion as a service with a pay-per-token model. In this case, you can embed your contacts as they are and run a nearest neighbors search (we call this process semantic similarity). The two records above will most certainly have a very high similarity score (e.g., greater than 0.85). Then we just show these two records to an LLM, and we ask it if the two records refer to the same entity or not; take this prompt as an example:

Consider the following two entities. Decide if they represent the same person or not.

Entity 1:

First Name: Nikolaos
Last Name: Vasiloglou
Email: vasiloglou@gmail.com
Company: [empty] Entity 2:
First Name: Nik
Last Name: Vasiloglou
Email: nik.vasiloglou@relational.ai
Company: RelationalAI Should these two entities be linked? Answer "yes" or "no."

If yes, we can further ask for the normalized linked record, including first name, last name, email, and company.

This is what we get:

Normalized record

First name: Nikolaos
Last name: Vasiloglou
Company: RelationalAI
emails:nik.vasiloglou@relational.ai, vasiloglou@gmail.com

LLMs have the common knowledge to identify possible first and last names and have seen a lot of email patterns. The solution is simple and clean.

The embedding cost is marginally zero, and LLM completions are very cheap. Moreover, the cloud infrastructure offers a high throughput of embedding generation and LLM completions, much higher than a single machine could give.

Security and privacy are always a concern when using cloud AI services, as data often leaves the secure enterprise environment. But what if your cloud database could offer all GenAI services in-house? Below is a code snippet showcasing how to perform entity linking with the RelationalAI native app, which operates within Snowflake's secure environment and utilizes Snowflake Cortex.

import relationalai as rai

from relationalai.std.graphs import Graph
from cortex_integration import generate, semantic_similarity

model = rai.Model("EntityResolution", format="snowpark")

User = model.Type("User", source="RAI_DATASETS. identity_resolution.users")

users_graph = Graph(model, undirected=True, weighted=True)
Node, Edge = users_graph.Node, users_graph.Edge

Node.extend(User, label=User.name, id=User.id)

prompt_template = """
    Consider the following two entities. Decide if they represent the same person or not. Your answer should be either "yes" or "no." No other explanation is needed.

    Entity 1:
    {summary1}

    Entity 2:
    {summary2}

    Should these two entities be linked? Answer "yes" or "no."
    """


with model.rule():
    user1 = User()
    user2 = User()

    # just make sure users are different
    user1 < user2

    score = semantic_similarity(user1.summary, user2.summary)

    # only consider semantically similar entities to minimize the LLM generations
    score > 0.85

    # let the LLM determine if the entities should be linked
    generate(prompt_template, user1.summary, user2.summary) == "Yes"

    # if the entities should be linked, add an edge between them
    Edge.add(user1, user2, weight=score)

The combination of Snowflake and RelationalAI brings the best of both worlds: simplicity and security. RelationalAI's declarative query language aligns with the vision of "Declarative Machine Learning," as highlighted by Chris Ré, enabling users to focus on what they want to achieve while leveraging the secure and scalable foundation provided by Snowflake. This approach helps simplify machine learning tasks without compromising data security within the enterprise perimeter. In summary, cloud-native AI services are a game changer to common problems like entity linking. They provide super high computational power on demand at a very low cost, enabling clean and simple solutions.