Secure RAG for SQLAlchemy and pgvector

We’ve spoken with nearly 100 engineering teams about their AI projects since the start of the year. Most have shipped something…to satisfy their CEOs, if nothing else. But they’re really just dipping their toes in the water, starting with low-risk areas like internal tools, autofill features, or Q&A over public knowledge bases. 

The world wants to ship more with LLMs, but most people who work in real companies are afraid of doing something wrong, especially leaking private data. 

So, today we’re launching the Oso SQLAlchemy integration. Python is the dominant language for building AI pipelines, and SQLAlchemy remains the most widely used ORM in that ecosystem. Integrating with SQLAlchemy makes Oso a drop-in solution for secure RAG applications. With Oso, developers and security engineers define their authorization logic once. Oso then converts that logic to SQLAlchemy filters, which you get automatically applied to every database query—including those in RAG pipelines using pgvector.

This gives CTOs, CISOs, and their engineering teams the confidence to ship secure, permissions-aware RAG applications by providing:

1. Security: A unified permissions layer that ensures the LLM only sees the data it should for any given user.
2. Visibility: The logic is in one place – easy to understand and change.
3. Reusability: The ability to apply this logic across services and databases.

Here’s why we built this capability, how it works, and how you can use it.

‍

Why is Authorization Hard in LLMs

In agentic and search problems such as RAG, source data is typically broken up into smaller chunks, which are then converted to embeddings – numerical representations of the text used for similarity search. These chunks often contain sensitive business data like documents, customer records, message threads, or support tickets.
‍
Applications usually have complex rules for who can access this kind of data: users may get access through role inheritance in an org chart, a link someone shared with them, or a related resource they have access to. You need to preserve and honor this authorization context during the retrieval process to ensure users only see embeddings that they’re authorized to access.

This query-to-retrieval flow is common across many agent orchestration frameworks and RAG systems:
‍
1. Embed the user query as a vector in the vector store.

2. Retrieve similar chunks using vector similarity.

3. Pass those chunks to the LLM for generation or summarization.

Our focus today is step 2: retrieval. Here, you need to apply the authorization logic from your core application – the rules that describe who is allowed to do and see what – over those chunks in order to ensure: a) that you only pass data to the LLM that the user is allowed to see, so b) the LLM’s response only references data that the user is allowed to see.

What’s so hard about this?

This kind of permissions logic is not typically stored in a single place that can be readily and holistically applied to new workloads, like RAG. By contrast, it’s more commonly spread across multiple files, libraries, and languages, a mix of application code and SQL logic. Often it’s comingled with business logic and other forms of state validation, making it hard to tease out so you can use it elsewhere. Lastly, without a single layer of enforcement, it’s hard to be sure that every request is getting properly checked and scoped down. This does not give security teams the warm and fuzzies.

While many looking into permissions for AI are reaching for OAuth, they quickly find that it’s a dead end. While OAuth can provide a way to control what actions one can take, it has no visibility into the underlying resources or data. To ensure your RAG app doesn’t leak private data, you need something else. This work has to happen in your code.
‍

Secure, In-Query Filtering with Oso and SQLAlchemy

The Oso SQLAlchemy integration enforces your authorization logic by converting it to SQL filters. Here’s how it works:

1. You define your authorization logic once in Polar, our DSL.
2. You annotate your SQLAlchemy models with Oso’s decorators (oso.Resource, oso.relation(), and oso.attribute()), mapping them to roles, relationships, and attributes from your policy.
3. At runtime, Oso uses your authorization logic (e.g., the owner of a resource can delete it) and actual data (e.g., role assignments, resource ownership) to generate SQL filters, which get passed through the ORM to Postgres. For a deeper dive on how this works, see A Logic Language for Distributed SQL Queries.
4. These filters are applied automatically through the .authorized() method on SQLAlchemy’s Select and Query classes, whether you’re querying structured data or RAG embeddings with pgvector.
   ‍

Built with Extensibility in Mind

We’re eager to share the Oso SQLAlchemy integration with the community. Feedback has been overwhelmingly positive thus far. For instance, when we met with Mike Bayer, the creator of SQLAlchemy, his first reaction was, “Wow, great news!”

We’re grateful for his feedback on the design, including suggestions like using with_loader_criteria for more idiomatic query integration.

Try it out:

- Check out the repository: it includes the library code and usage examples.
- Review the API reference.

While this release focuses on SQLAlchemy, we’re also exploring integrations for other ORMs like Prisma and Django, and data layers like MongoDB and Elasticsearch. If you’re working with one of those stacks, we’d love to hear from you.

You can also join our Slack community to connect with other engineers building RAG pipelines on SQLAlchemy.