docs: add Elasticsearch and OpenSearch sink guides - #38025
Conversation
Adds guides for sinking Materialize results to Elasticsearch and OpenSearch via the Kafka sink and Kafka Connect, using the perfect-embedding transform to keep vector embeddings fresh.
| [Kafka sink](/sql/create-sink/kafka/) writes the results to a Kafka topic. | ||
| Kafka Connect reads that topic and writes the documents to Elasticsearch. | ||
|
|
||
| The [`perfect-embedding`](https://github.com/MaterializeInc/perfect-embedding) |
There was a problem hiding this comment.
@claude: Minor note, but I'd include 1-2 lines about this transform. Something like: "in this guide, we'll also use perfect-embedding, a Kafka Connect SMT (single message transform) that we developed. Perfect-embedding runs ...`
Do the same on the other page as well
| weight: 20 | ||
| --- | ||
|
|
||
| This guide shows how to send results from Materialize to Elasticsearch. A |
There was a problem hiding this comment.
@claude let's include one line here about how you can use this to keep search indexes in Elasticsearch up to date - leverage the copy we wrote on our website here: https://materialize.com/solutions/ai-native-search/
| @@ -0,0 +1,60 @@ | |||
| The examples in this guide build a search document for an article catalog tracking its content and page views. | |||
There was a problem hiding this comment.
Oh ... we don't use shared-content anymore. Instead, we use headless content (which is better because when you're running locally, and you modify headless content, it renders your changes whereas with shared content ... you actually have to restart hugo).
{{% include-headless "/headless/blah" %}}
3 participants
Summary
Test plan