Overview
Retrieval-Augmented Generation (RAG) enhances LLM responses by injecting relevant context before generation. Trix provides the memory layer that makes RAG simple, fast, and effective.
Unlike traditional RAG systems that require managing vector databases, embeddings, and complex pipelines, Trix handles all of this automatically. You store memories, query them semantically, and inject the results into your prompts.
Basic RAG Pattern
The fundamental RAG pattern with Trix involves three steps:
- Query - Search for relevant memories based on the user's input
- Augment - Add retrieved memories to the LLM prompt
- Generate - Let the LLM respond with full context
import { Trix } from '@trixdb/client';
import OpenAI from 'openai';
const client = new Trix({ apiKey: process.env.TRIX_API_KEY });
const openai = new OpenAI();
async function ragQuery(userQuery: string, spaceId: string) {
const memories = await client.memories.list({
q: userQuery, spaceId, limit: 5
});
const context = memories.data.map(m => m.content).join('\n');
const response = await openai.chat.completions.create({
model: 'gpt-4',
messages: [
{ role: 'system', content: `Answer based on this context:\n${context}` },
{ role: 'user', content: userQuery }
]
});
return response.choices[0].message.content;
}Context Window Management
LLMs have limited context windows. With Trix, you control exactly how much context to retrieve and how to prioritize it.
async function managedContextRAG(query: string, spaceId: string) {
const MAX_CONTEXT_TOKENS = 4000;
const AVG_CHARS_PER_TOKEN = 4;
const maxChars = MAX_CONTEXT_TOKENS * AVG_CHARS_PER_TOKEN;
const memories = await client.memories.list({
q: query, spaceId, limit: 20
});
let context = '';
let usedMemories = [];
for (const memory of memories.data) {
if ((context + memory.content).length > maxChars) break;
context += memory.content + '\n\n';
usedMemories.push(memory);
}
return { context, usedMemories, totalFound: memories.data.length };
}Trix returns memories sorted by relevance score. The most relevant memories come first, so truncating from the end preserves the best context.
Hybrid Search Patterns
Combine different search strategies for more comprehensive retrieval. Use tags and metadata to filter, then semantic search to rank.
async function hybridSearch(query: string, spaceId: string) {
const semantic = await client.memories.list({ q: query, spaceId, limit: 5 });
const tagged = await client.memories.list({
q: query, spaceId, tags: ['important', 'verified'], limit: 5
});
const recent = await client.memories.list({
q: query, spaceId, limit: 5, sort: 'created_at:desc'
});
const seen = new Set();
const merged = [];
for (const m of [...semantic.data, ...tagged.data, ...recent.data]) {
if (!seen.has(m.id)) {
seen.add(m.id);
merged.push(m);
}
}
return merged;
}Query Expansion
Trix now does much of this for you: for complex questions it plans multi-step recall and fuses several sub-queries with reciprocal rank fusion (RRF) server-side. The DIY pattern below is still useful for domain-specific expansion, but it's no longer required for good recall — see Search.
Improve recall by expanding the user's query before searching. Use the LLM to generate related terms and concepts.
async function expandedSearch(userQuery: string, spaceId: string) {
const expansion = await openai.chat.completions.create({
model: 'gpt-4',
messages: [
{ role: 'system', content: `Generate 3 alternative phrasings for this query.
Return as JSON array of strings. Be concise.` },
{ role: 'user', content: userQuery }
],
response_format: { type: 'json_object' }
});
const { queries } = JSON.parse(expansion.choices[0].message.content);
const allQueries = [userQuery, ...queries];
const results = await Promise.all(
allQueries.map(q => client.memories.list({ q, spaceId, limit: 3 }))
);
const seen = new Set();
return results
.flatMap(r => r.data)
.filter(m => {
if (seen.has(m.id)) return false;
seen.add(m.id);
return true;
});
}Query expansion adds latency due to the extra LLM call. Use it for complex queries where recall is more important than speed.
Metadata Filtering
Use metadata to pre-filter memories before semantic search. This narrows results to the most relevant subset.
await client.memories.create({
content: 'Q4 revenue exceeded projections by 15%',
spaceId: companySpaceId,
metadata: {
type: 'financial',
quarter: 'Q4',
year: 2024,
confidential: true
}
});
async function filteredSearch(query: string, filters: object) {
return client.memories.list({
q: query, spaceId: companySpaceId, metadata: filters, limit: 10
});
}
const financialQ4 = await filteredSearch('revenue', { type: 'financial', quarter: 'Q4' });
const nonConfidential = await filteredSearch('company updates', { confidential: false });Re-ranking Results
Trix can re-rank with a dedicated cross-encoder as part of retrieval, so you often don't need a hand-rolled LLM re-ranker. The pattern below remains a good escape hatch when you want fully custom scoring.
For critical applications, use a second-pass re-ranking to improve precision. The LLM evaluates each result for relevance.
async function rerankResults(
query: string, memories: Memory[], topK: number = 3
) {
const reranked = await openai.chat.completions.create({
model: 'gpt-4',
messages: [
{ role: 'system', content: `Rate each document's relevance to the query (0-10).
Return JSON: { "rankings": [{ "index": 0, "score": 8 }, ...] }` },
{ role: 'user', content: JSON.stringify({
query,
documents: memories.map((m, i) => ({ index: i, content: m.content }))
}) }
],
response_format: { type: 'json_object' }
});
const { rankings } = JSON.parse(reranked.choices[0].message.content);
return rankings
.sort((a, b) => b.score - a.score)
.slice(0, topK)
.map(r => memories[r.index]);
}Re-ranking works best when you fetch more candidates than needed (e.g., fetch 20, re-rank to top 5). This balances recall with precision.
Chunking Strategies
How you chunk content before storing affects retrieval quality. Different content types benefit from different strategies.
- Sentence-level - Best for facts and discrete pieces of information
- Paragraph-level - Good for narrative content that needs context
- Semantic chunking - Split on topic boundaries for natural groupings
- Sliding window - Overlapping chunks preserve context across boundaries
function chunkDocument(text: string, chunkSize = 500, overlap = 50) {
const sentences = text.split(/(?<=[.!?])\s+/);
const chunks = [];
let currentChunk = '';
for (const sentence of sentences) {
if ((currentChunk + sentence).length > chunkSize && currentChunk) {
chunks.push(currentChunk.trim());
currentChunk = currentChunk.slice(-overlap) + sentence;
} else {
currentChunk += ' ' + sentence;
}
}
if (currentChunk) chunks.push(currentChunk.trim());
return chunks;
}
async function ingestDocument(content: string, docId: string, spaceId: string) {
const chunks = chunkDocument(content);
await Promise.all(chunks.map((chunk, i) =>
client.memories.create({
content: chunk,
spaceId,
metadata: { docId, chunkIndex: i, totalChunks: chunks.length }
})
));
}Production-Ready Pattern
A complete RAG implementation with error handling, logging, and fallbacks.
class RAGPipeline {
constructor(
private client: Trix,
private openai: OpenAI,
private config: RAGConfig
) {}
async query(userQuery: string, spaceId: string): Promise<RAGResponse> {
const startTime = Date.now();
try {
let memories = await this.retrieve(userQuery, spaceId);
if (memories.length === 0 && this.config.expandOnEmpty) {
memories = await this.expandedRetrieve(userQuery, spaceId);
}
const context = this.buildContext(memories);
const response = await this.generate(userQuery, context);
return {
answer: response,
sources: memories.map(m => m.id),
latencyMs: Date.now() - startTime
};
} catch (error) {
console.error('RAG retrieval failed:', error);
const response = await this.generate(userQuery, '');
return {
answer: response, sources: [],
latencyMs: Date.now() - startTime, fallback: true
};
}
}
private buildContext(memories: Memory[]): string {
let context = '';
for (const m of memories) {
if (context.length + m.content.length > this.config.maxContextChars) break;
context += m.content + '\n\n';
}
return context;
}
}Best Practices
- Start simple - Basic RAG often works well. Add complexity only when needed.
- Measure retrieval quality - Track if retrieved memories are actually used in responses.
- Use appropriate chunk sizes - Too small loses context, too large dilutes relevance.
- Set similarity thresholds - Filter out low-relevance results to reduce noise.
- Cache frequent queries - RAG results for common queries can be cached.
- Monitor latency - Retrieval should add minimal time to your response.
Trix's semantic search typically returns results in under 100ms, making it suitable for real-time RAG applications.