\n","children":[{"type":"text","text":""}]},{"type":"mdxJsxFlowElement","name":"YouTubeVideo","children":[{"type":"text","text":""}],"props":{"videoId":"jYXJEiVFDv0"}},{"type":"p","children":[{"type":"text","text":"We are building a Relational Knowledge Graph System. We have a big vision for\nthis product, and you can learn more about that vision in other videos on our\n"},{"type":"a","url":"/","title":null,"children":[{"type":"text","text":"website"}]},{"type":"text","text":"."}]},{"type":"p","children":[{"type":"text","text":"Today we are going to focus on Data Applications built on Knowledge Graphs."}]},{"type":"p","children":[{"type":"text","text":"We're going to walk through some highlights of a demo that uses a Knowledge\nGraph to solve a business problem."}]},{"type":"p","children":[{"type":"text","text":"This demo is called the \"Knowledge Graph to Learn Knowledge Graphs\",\nor KGLKG. It parallels my own journey from programming in imperative languages\nlike Java, C++, and Python to RAI's declarative language, Rel. For many years, I\nused those legacy languages plus SQL in an application-centric database-oriented\nway of solving business problems. But today I am using Rel and a data-centric\nbusiness modeling approach to creating business solutions."}]},{"type":"p","children":[{"type":"text","text":"If you have seen the Overview video, or had a presentation from our Sales team,\nyou will recognize this reference architecture diagram for Data Apps built on\nthe RAI platform."}]},{"type":"p","children":[{"type":"text","text":"Here is how the KGLKG Data App fits into this architecture."}]},{"type":"p","children":[{"type":"text","text":"A full demo would walk through these components, but we’re going to jump ahead\nto the demo itself."}]},{"type":"p","children":[{"type":"text","text":"A Jupyter notebook will serve as the front-end for today's demo."}]},{"type":"p","children":[{"type":"text","text":"The business problem revolves around choosing the most relevant learning\nmaterials to bring a new Sales or Sales Engineering hire up to speed. Documents\nare suggested based on their conceptual content, quality, degree of difficulty,\nand appropriateness for a sequenced learning plan."}]},{"type":"p","children":[{"type":"text","text":"We model the business problem with this Knowledge Graph. Let’s build it up piece\nby piece."}]},{"type":"p","children":[{"type":"text","text":"We begin with the core nodes and relationships (or edges) that center around the\nlearning materials and the concepts or topics they contain. These learning\nmaterials are PDFs and HTML pages like blog posts, e-zine articles, videos, and\nso on. We’re going to focus on PDFs and HTMLs."}]},{"type":"p","children":[{"type":"text","text":"We ingest a lemma CSV containing document names, concept names, and concept\nweights (the number of occurrences of that concept in the document)."}]},{"type":"p","children":[{"type":"text","text":"We define relations in Rel for the document and concept nodes. These are based\non the lemma CSV. Here are the documents…"}]},{"type":"p","children":[{"type":"text","text":"The "},{"type":"text","text":"About","code":true},{"type":"text","text":" edges link the Document and Concept nodes. A document is about some\nset of concepts. And each concept has some number of documents that mention it.\nEach such "},{"type":"text","text":"About","code":true},{"type":"text","text":" relationship has a weight attribute, the number of times that\nconcept appears in that document."}]},{"type":"p","children":[{"type":"text","text":"We can list all the "},{"type":"text","text":"About","code":true},{"type":"text","text":" edges or "},{"type":"text","text":"about_weight","code":true},{"type":"text","text":" attributes, but here let's\nquery the "},{"type":"text","text":"about_weight","code":true},{"type":"text","text":" attribute and use a regular expression to find the\ndocuments and "},{"type":"text","text":"about_weights","code":true},{"type":"text","text":" for concept words that contain the substring,\n\"graph\"."}]},{"type":"p","children":[{"type":"text","text":"Here's the Knowledge Graph fragment we've created so far annotated with example\ndata."}]},{"type":"p","children":[{"type":"text","text":"In addition to querying attributes from the loaded data, we can compute\nknowledge from the nodes and relationships."}]},{"type":"p","children":[{"type":"text","text":"These computations can be executed as runtime queries."}]},{"type":"p","children":[{"type":"text","text":"Or, they can be generalized and declaratively defined as relations so that this\ncomputed knowledge becomes part of our Knowledge Graph, available for simple\nquery."}]},{"type":"p","children":[{"type":"text","text":"Our diagram highlights the new computed attributes added to our Knowledge Graph.\nThese are automatically recomputed as the underlying data changes, say when a\nnew document and its concepts are added to the graph. Contrast this with other\ngraph databases which require you to explicitly run queries to recompute\nattributes that may be affected by data updates."}]},{"type":"p","children":[{"type":"text","text":"To suggest documents based on their content, we compute the \"focus\" of\neach document on each of the concepts it mentions. The "},{"type":"text","text":"about_focus","code":true},{"type":"text","text":" attribute\nbecomes a queryable part of our knowledge graph. But for convenience, we create\nan additional relation, called \"suggested\" to get us a Top-N list of\ndocuments for a specified concept."}]},{"type":"p","children":[{"type":"text","text":"The definition shown here is unbounded, too generalized to be pre-computed. So\nwe mark it with \"@inline\" to defer evaluation of the relation until it\nis used in a query for a specific concept. You might even think of the suggested\nrelation as similar to a procedural language function."}]},{"type":"p","children":[{"type":"text","text":"Here's a query that joins the top-N list for \"graph\" with the top-N\nlist for \"ai\" to get a top-N list of documents about BOTH topics."}]},{"type":"p","children":[{"type":"text","text":"So far we have based our suggestions purely on statistical analysis of the\ndocuments. But now we want to include knowledge from Reviewers who rank and rate\nthese documents."}]},{"type":"p","children":[{"type":"text","text":"We add a Person node with a dynamic role attribute that can track their status\nas learner, employee, or curator of the learning materials."}]},{"type":"p","children":[{"type":"text","text":"The full demo shows how we dynamically add data, perform ELT, and use integrity\nconstraints to guarantee our data conforms to our business rules. Right now,\nwe're going to skip ahead to the Reviewer role."}]},{"type":"p","children":[{"type":"text","text":"Any employee can review documents, and they are not confined to the curated\nlibrary of materials. In fact, reviewing documents outside the curated library\nis how new material is found for the library."}]},{"type":"p","children":[{"type":"text","text":"In this example, we have two reviewers, Ben and Steve. Ben leads a sales team\nand reviews documents with an eye towards onboarding new sales hires. He tracks\nhis reading in a Google Sheet called Sales Onboarding Plan, or SOP for short."}]},{"type":"p","children":[{"type":"text","text":"Steve leads a sales engineering team and reviews documents with an eye towards\nonboarding new sales engineering hires. He tracks his reading in a Google Sheet\ncalled Hersker's Learning Curve, or HLC for short."}]},{"type":"p","children":[{"type":"text","text":"Reflecting the real world, there was no standard format for these tracking\nsheets, so Ben and Steve each created different formats that must be reconciled\nfor our analysis. In his sheet, Ben created document titles that were linked to\nthe URL. Steve used separate columns for the title and the URL. Both used some\nGoogle Drive URLs which convey no human-readable information, but which can be\nresolved to the actual on-drive file names."}]},{"type":"p","children":[{"type":"text","text":"We define brief and full-detailed definitions for the input CSVs for each\nreviewer’s Google Sheet. Here's the brief data for each."}]},{"type":"p","children":[{"type":"text","text":"The detailed views include extracted URLs and Names generated during data\npreparation. But the data ingest process left some entries we don't want."}]},{"type":"p","children":[{"type":"text","text":"For example, there are many empty strings in the "},{"type":"text","text":"extractedName","code":true},{"type":"text","text":" relation which\nwe want to eliminate as they violate 6NF."}]},{"type":"p","children":[{"type":"text","text":"So, we perform some ELT to clean up our reviewer data."}]},{"type":"p","children":[{"type":"text","text":"Now we have a clean "},{"type":"text","text":"extractedName","code":true},{"type":"text","text":" relation."}]},{"type":"p","children":[{"type":"text","text":"To reconcile these reviewer lists we want to match names and find the\nintersections between the two lists and with the curated library. But we will\nexplore these next steps in other videos."}]}],"_content_source":{"queryId":"src/content/resources/demo-highlights.mdx","path":["resource","body"]}},"_content_source":{"queryId":"src/content/resources/demo-highlights.mdx","path":["resource"]}}},"errors":null,"query":"\n query resource($relativePath: String!) {\n resource(relativePath: $relativePath) {\n ... on Document {\n _sys {\n filename\n basename\n breadcrumbs\n path\n relativePath\n extension\n }\n id\n }\n ...ResourceParts\n }\n}\n \n fragment ResourceParts on Resource {\n __typename\n title\n description\n date\n image\n categories\n authors {\n __typename\n name\n link\n }\n seo {\n __typename\n keywords\n description\n image\n image_alt\n canonical_url\n author\n published\n modified\n language\n robots\n site_name\n content_type\n }\n body\n}\n ","variables":{"relativePath":"demo-highlights.mdx"}},"src/content/meta/meta.md":{"data":{"meta":{"_sys":{"filename":"meta","basename":"meta.md","breadcrumbs":["meta"],"path":"src/content/meta/meta.md","relativePath":"meta.md","extension":".md"},"id":"src/content/meta/meta.md","__typename":"Meta","banner":{"__typename":"MetaBanner","enabled":true,"content":{"type":"root","children":[{"type":"p","children":[{"type":"text","text":"Check out "},{"type":"a","url":"/resources/highlights-of-relationalai-at-snowflake-data-cloud-summit-2024","title":"SF summit highlights","children":[{"type":"text","text":"highlights"}]},{"type":"text","text":" of RelationalAI at "},{"type":"text","text":"Snowflake's Data Cloud Summit 2024!","bold":true}]}],"_content_source":{"queryId":"src/content/meta/meta.md","path":["meta","banner","content"]}},"_content_source":{"queryId":"src/content/meta/meta.md","path":["meta","banner"]}},"header":{"__typename":"MetaHeader","links":[{"__typename":"MetaHeaderLinks","text":"Product","url":"/product","style":"default","_content_source":{"queryId":"src/content/meta/meta.md","path":["meta","header","links",0]}},{"__typename":"MetaHeaderLinks","text":"Company","url":"/company","style":"default","_content_source":{"queryId":"src/content/meta/meta.md","path":["meta","header","links",1]}},{"__typename":"MetaHeaderLinks","text":"Docs","url":"/docs","style":"default","_content_source":{"queryId":"src/content/meta/meta.md","path":["meta","header","links",2]}},{"__typename":"MetaHeaderLinks","text":"Resources","url":"/resources/all/1","style":"default","_content_source":{"queryId":"src/content/meta/meta.md","path":["meta","header","links",3]}},{"__typename":"MetaHeaderLinks","text":"Get Started","url":"/get-started","style":"cta","_content_source":{"queryId":"src/content/meta/meta.md","path":["meta","header","links",4]}}],"_content_source":{"queryId":"src/content/meta/meta.md","path":["meta","header"]}},"footer":{"__typename":"MetaFooter","sections":[{"__typename":"MetaFooterSections","name":"Product","links":[{"__typename":"MetaFooterSectionsLinks","text":"Overview","url":"/product","_content_source":{"queryId":"src/content/meta/meta.md","path":["meta","footer","sections",0,"links",0]}},{"__typename":"MetaFooterSectionsLinks","text":"Use Cases","url":"/product#for-problems-that-matter","_content_source":{"queryId":"src/content/meta/meta.md","path":["meta","footer","sections",0,"links",1]}},{"__typename":"MetaFooterSectionsLinks","text":"Capabilities","url":"/product#a-new-toolset","_content_source":{"queryId":"src/content/meta/meta.md","path":["meta","footer","sections",0,"links",2]}}],"_content_source":{"queryId":"src/content/meta/meta.md","path":["meta","footer","sections",0]}},{"__typename":"MetaFooterSections","name":"Resources","links":[{"__typename":"MetaFooterSectionsLinks","text":"Documentation","url":"/docs/getting_started","_content_source":{"queryId":"src/content/meta/meta.md","path":["meta","footer","sections",1,"links",0]}},{"__typename":"MetaFooterSectionsLinks","text":"News","url":"/resources/news/1","_content_source":{"queryId":"src/content/meta/meta.md","path":["meta","footer","sections",1,"links",1]}},{"__typename":"MetaFooterSectionsLinks","text":"Research","url":"/resources/research/1","_content_source":{"queryId":"src/content/meta/meta.md","path":["meta","footer","sections",1,"links",2]}},{"__typename":"MetaFooterSectionsLinks","text":"Releases","url":"/resources/releases/1","_content_source":{"queryId":"src/content/meta/meta.md","path":["meta","footer","sections",1,"links",3]}}],"_content_source":{"queryId":"src/content/meta/meta.md","path":["meta","footer","sections",1]}},{"__typename":"MetaFooterSections","name":"About Us","links":[{"__typename":"MetaFooterSectionsLinks","text":"Our Company","url":"/company","_content_source":{"queryId":"src/content/meta/meta.md","path":["meta","footer","sections",2,"links",0]}},{"__typename":"MetaFooterSectionsLinks","text":"Contact Us","url":"/get-started","_content_source":{"queryId":"src/content/meta/meta.md","path":["meta","footer","sections",2,"links",1]}},{"__typename":"MetaFooterSectionsLinks","text":"Careers","url":"/careers","_content_source":{"queryId":"src/content/meta/meta.md","path":["meta","footer","sections",2,"links",2]}},{"__typename":"MetaFooterSectionsLinks","text":"Legal","url":"/legal","_content_source":{"queryId":"src/content/meta/meta.md","path":["meta","footer","sections",2,"links",3]}},{"__typename":"MetaFooterSectionsLinks","text":"GDPR","url":"/gdpr","_content_source":{"queryId":"src/content/meta/meta.md","path":["meta","footer","sections",2,"links",4]}},{"__typename":"MetaFooterSectionsLinks","text":"Security & Trust","url":"https://trust.relational.ai/","_content_source":{"queryId":"src/content/meta/meta.md","path":["meta","footer","sections",2,"links",5]}}],"_content_source":{"queryId":"src/content/meta/meta.md","path":["meta","footer","sections",2]}}],"socials":[{"__typename":"MetaFooterSocials","text":"GitHub","url":"https://github.com/RelationalAI","icon":"https://assets.tina.io/91d76337-e55d-4722-acb5-3106adb895b6/img/logos/github.png","_content_source":{"queryId":"src/content/meta/meta.md","path":["meta","footer","socials",0]}},{"__typename":"MetaFooterSocials","text":"LinkedIn","url":"https://www.linkedin.com/company/relationalai/about","icon":"https://assets.tina.io/91d76337-e55d-4722-acb5-3106adb895b6/img/logos/linkedin.png","_content_source":{"queryId":"src/content/meta/meta.md","path":["meta","footer","socials",1]}},{"__typename":"MetaFooterSocials","text":"Twitter","url":"https://twitter.com/relationalai","icon":"https://assets.tina.io/91d76337-e55d-4722-acb5-3106adb895b6/img/logos/twitter.png","_content_source":{"queryId":"src/content/meta/meta.md","path":["meta","footer","socials",2]}}],"_content_source":{"queryId":"src/content/meta/meta.md","path":["meta","footer"]}},"_content_source":{"queryId":"src/content/meta/meta.md","path":["meta"]}}},"errors":null,"query":"\n query meta($relativePath: String!) {\n meta(relativePath: $relativePath) {\n ... on Document {\n _sys {\n filename\n basename\n breadcrumbs\n path\n relativePath\n extension\n }\n id\n }\n ...MetaParts\n }\n}\n \n fragment MetaParts on Meta {\n __typename\n banner {\n __typename\n enabled\n content\n }\n header {\n __typename\n links {\n __typename\n text\n url\n style\n }\n }\n footer {\n __typename\n sections {\n __typename\n name\n links {\n __typename\n text\n url\n }\n }\n socials {\n __typename\n text\n url\n icon\n }\n }\n}\n ","variables":{"relativePath":"./meta.md"}}};
globalThis.tina_info = tina;
})();
Demo Highlights · RelationalAI
Check out highlights of RelationalAI at Snowflake's Data Cloud Summit 2024!
We are building a Relational Knowledge Graph System. We have a big vision for
this product, and you can learn more about that vision in other videos on our
website.
Today we are going to focus on Data Applications built on Knowledge Graphs.
We're going to walk through some highlights of a demo that uses a Knowledge
Graph to solve a business problem.
This demo is called the "Knowledge Graph to Learn Knowledge Graphs",
or KGLKG. It parallels my own journey from programming in imperative languages
like Java, C++, and Python to RAI's declarative language, Rel. For many years, I
used those legacy languages plus SQL in an application-centric database-oriented
way of solving business problems. But today I am using Rel and a data-centric
business modeling approach to creating business solutions.
If you have seen the Overview video, or had a presentation from our Sales team,
you will recognize this reference architecture diagram for Data Apps built on
the RAI platform.
Here is how the KGLKG Data App fits into this architecture.
A full demo would walk through these components, but we’re going to jump ahead
to the demo itself.
A Jupyter notebook will serve as the front-end for today's demo.
The business problem revolves around choosing the most relevant learning
materials to bring a new Sales or Sales Engineering hire up to speed. Documents
are suggested based on their conceptual content, quality, degree of difficulty,
and appropriateness for a sequenced learning plan.
We model the business problem with this Knowledge Graph. Let’s build it up piece
by piece.
We begin with the core nodes and relationships (or edges) that center around the
learning materials and the concepts or topics they contain. These learning
materials are PDFs and HTML pages like blog posts, e-zine articles, videos, and
so on. We’re going to focus on PDFs and HTMLs.
We ingest a lemma CSV containing document names, concept names, and concept
weights (the number of occurrences of that concept in the document).
We define relations in Rel for the document and concept nodes. These are based
on the lemma CSV. Here are the documents…
The About edges link the Document and Concept nodes. A document is about some
set of concepts. And each concept has some number of documents that mention it.
Each such About relationship has a weight attribute, the number of times that
concept appears in that document.
We can list all the About edges or about_weight attributes, but here let's
query the about_weight attribute and use a regular expression to find the
documents and about_weights for concept words that contain the substring,
"graph".
Here's the Knowledge Graph fragment we've created so far annotated with example
data.
In addition to querying attributes from the loaded data, we can compute
knowledge from the nodes and relationships.
These computations can be executed as runtime queries.
Or, they can be generalized and declaratively defined as relations so that this
computed knowledge becomes part of our Knowledge Graph, available for simple
query.
Our diagram highlights the new computed attributes added to our Knowledge Graph.
These are automatically recomputed as the underlying data changes, say when a
new document and its concepts are added to the graph. Contrast this with other
graph databases which require you to explicitly run queries to recompute
attributes that may be affected by data updates.
To suggest documents based on their content, we compute the "focus" of
each document on each of the concepts it mentions. The about_focus attribute
becomes a queryable part of our knowledge graph. But for convenience, we create
an additional relation, called "suggested" to get us a Top-N list of
documents for a specified concept.
The definition shown here is unbounded, too generalized to be pre-computed. So
we mark it with "@inline" to defer evaluation of the relation until it
is used in a query for a specific concept. You might even think of the suggested
relation as similar to a procedural language function.
Here's a query that joins the top-N list for "graph" with the top-N
list for "ai" to get a top-N list of documents about BOTH topics.
So far we have based our suggestions purely on statistical analysis of the
documents. But now we want to include knowledge from Reviewers who rank and rate
these documents.
We add a Person node with a dynamic role attribute that can track their status
as learner, employee, or curator of the learning materials.
The full demo shows how we dynamically add data, perform ELT, and use integrity
constraints to guarantee our data conforms to our business rules. Right now,
we're going to skip ahead to the Reviewer role.
Any employee can review documents, and they are not confined to the curated
library of materials. In fact, reviewing documents outside the curated library
is how new material is found for the library.
In this example, we have two reviewers, Ben and Steve. Ben leads a sales team
and reviews documents with an eye towards onboarding new sales hires. He tracks
his reading in a Google Sheet called Sales Onboarding Plan, or SOP for short.
Steve leads a sales engineering team and reviews documents with an eye towards
onboarding new sales engineering hires. He tracks his reading in a Google Sheet
called Hersker's Learning Curve, or HLC for short.
Reflecting the real world, there was no standard format for these tracking
sheets, so Ben and Steve each created different formats that must be reconciled
for our analysis. In his sheet, Ben created document titles that were linked to
the URL. Steve used separate columns for the title and the URL. Both used some
Google Drive URLs which convey no human-readable information, but which can be
resolved to the actual on-drive file names.
We define brief and full-detailed definitions for the input CSVs for each
reviewer’s Google Sheet. Here's the brief data for each.
The detailed views include extracted URLs and Names generated during data
preparation. But the data ingest process left some entries we don't want.
For example, there are many empty strings in the extractedName relation which
we want to eliminate as they violate 6NF.
So, we perform some ELT to clean up our reviewer data.
Now we have a clean extractedName relation.
To reconcile these reviewer lists we want to match names and find the
intersections between the two lists and with the curated library. But we will
explore these next steps in other videos.