Preview environment - you also see content that isn't published yet.
5 min

What Else Is Out There

RAG isn't only text. What multimodal RAG changes, why MCP servers are a fascinating source, and which directions a later tutorial will dig into.

We built a standard RAG stack: text corpus, chunking, embeddings, vector DB, retrieval, optional rerank, LLM answer. That alone already covers a surprising range of use cases.

Still, it's the entry point. Three paths to follow later.

Multimodal - RAG for anything that isn't text

Embedding models long ago outgrew plain text. Vision embedders place images into the same vector space as their text descriptions. Audio embedders cover spoken language. Code embedders cover source files.

The RAG pipeline barely changes:

  • Instead of "chunk Markdown", now "extract frames from a video" or "extract functions from a repo".
  • Instead of a text embedder, a matching multimodal model.
  • Retrieval and top-K stay the same.

Example applications: search inside your photo library, code search across a monorepo, answers to spoken questions from meeting transcripts.

RAG for MCP - tools instead of documents

A fun special case: your "corpus" isn't static documents - it's tool descriptions, e.g. from MCP servers (Model Context Protocol).

Instead of dumping all 200 available tools into the prompt, you embed the tool descriptions, retrieve the 5 most relevant for the current task via RAG, and only expose those as available. That saves a ton of context and turns models that can't reliably tool-call across many options into usable agents again.

The pattern is still young but already very productive - we'll work through a concrete build in a later tutorial.

Where from here?

What we didn't touch in v1 of this tutorial, but which is worth chasing:

  • Production-grade freshness and updates: document hashes, incremental re-embedding, hot reload.
  • Per-user corpora: every user uploads their own documents, all share the same stack.
  • Cluster-scale corpora: above millions of documents, swapping pgvector for Qdrant or Weaviate is worth a benchmark.
  • Agentic retrieval: the LLM picks its search strategy, paraphrases the question iteratively, combines multiple retrievers.

But for 90% of practical applications, what you understand now is enough. That's the biggest strength of RAG: the concept is simple, the leverage is enormous.

Discussion· no posts yet

Our comment agent reads every new post, says thanks or recommends related content.

Be the first voice - what do you think?

Sign in to join the discussion.

Sign in