> ## Documentation Index
> Fetch the complete documentation index at: https://botmesh.afaya.com.co/llms.txt
> Use this file to discover all available pages before exploring further.

# AI memory & RAG

Two ways to give the [AI plugin](https://botmesh.afaya.com.co/guide/ai.md) more context than the chat
history:

* **Long-term memory** remembers short facts about each person ("prefers
  Spanish", "lives in Bogotá") across conversations and restarts, and injects
  them into the autoreply.
* **RAG** (Retrieval Augmented Generation) finds the passages of your
  documents (FAQ, catalog, manual) that answer the question and injects them,
  numbered, so the model answers from them and cites them.

Both feed the system prompt of `replyInChat()` (and so of the autoresponder);
neither one fails a reply: if they error or time out, the turn goes on without
them.

## Long-term memory

```ts theme={null}
bot.plug(aiPlugin({
  provider: 'openai',
  model: 'gpt-5-mini',
  memory: {
    maxFacts: 50,      // oldest facts are dropped first
    ttl: '90d',        // each fact expires after 90 days (re-stating it renews it)
    autoExtract: true, // after each reply, the model extracts new facts (strict JSON)
  },
}))
```

### Where facts live

Facts are stored in the persistent user profile, [`ctx.user`](https://botmesh.afaya.com.co/guide/user-data),
under the namespace `'ai.memory'` (key `facts`), keyed by the canonical sender
id (`ctx.sender`). So:

* The same person keeps one memory across their DM and the groups they write
  in, and across restarts when the store is persistent (`MemoryStore` lives in
  RAM).
* It needs a store with user-data support (all the bundled stores have it).
* `bot.deleteSubject(userId)` erases it together with the rest of the profile.
* `/reset` clears the chat's AI context, **not** the long-term memory.

### What gets injected

With `memory` on, the user's facts are added to the system prompt of every
autoreply turn, as data, not instructions. **Groups are excluded by default**:
every member reads a group reply, so facts learned in private could leak
there. Turn `injectInGroups: true` on only if that is acceptable.

### Automatic extraction

With `autoExtract`, after each turn the plugin asks the model, **in the
background** (it never blocks the chat's queue), for durable facts about the
user. The answer must be exactly `{"facts": string[]}`: anything else (extra
keys, prose, non-string items) is discarded whole, so a confused or
prompt-injected model cannot write arbitrary data into the profile. The prompt
also tells the model to skip secrets and sensitive data (passwords, card or
account numbers, government ids, health data).

If the user's data is erased (`bot.deleteSubject()` or `forget(userId)`) while
an extraction is running, its facts are dropped: erased data never comes back.

`autoExtract` accepts `true` or an object:

| Field | Default | Description |
| - | - | - |
| `model` | the plugin's `model` | Model for the extraction call. |
| `maxTokens` | `1024` | Token budget of the extraction call (reasoning models spend part of it thinking; Ollama gets no cap unless you set it). |
| `maxFactsPerTurn` | `5` | Max new facts accepted per turn. |
| `instructions` | — | Extra guidance appended to the extraction prompt. |
| `filter` | — | `(fact) => boolean`: return `false` to drop a candidate (e.g. a PII filter). |

### `memory` options

| Field | Type | Default | Description |
| - | - | - | - |
| `type` | `'per-user'` | `'per-user'` | Only per-user memory exists (keyed by `ctx.sender`). |
| `maxFacts` | `number` | `50` | Facts kept per user; the oldest are dropped first. |
| `ttl` | `string \| number` | permanent | Lifetime of each fact (`'30d'`, `'12h'`, ms). Re-stating a fact renews it. |
| `maxFactLength` | `number` | `300` | Longer facts are rejected. `aiPlugin()` throws a `RangeError` for a value that is not a positive integer, and for a `ttl` or `maxFacts` that cannot work. |
| `inject` | `boolean` | `true` | Inject the facts into the system prompt. |
| `injectInGroups` | `boolean` | `false` | Also inject them when the user writes in a group. |
| `autoExtract` | `boolean \| object` | `false` | Background extraction (see above). |
| `namespace` | `string` | `'ai.memory'` | Profile namespace. |

`memory: true` uses the defaults.

### Reading and writing memory

`ctx.tools.ai` exposes the memory API. It works even without the `memory`
option (then nothing is injected or extracted automatically, but your code can
still read and write facts).

```ts theme={null}
bot.when('what do you know about me', async (ctx) => {
  const facts = await ctx.tools.ai.memory(ctx.sender)
  // ['Prefers Spanish', 'Lives in Bogotá']
  await ctx.reply(facts.join('\n') || 'Nothing yet')
})

const ai = bot.injectTools.ai
await ai.remember(userId, 'VIP customer')              // resolves with the facts that were new
await ai.remember(userId, ['Vegan', 'Allergic to nuts'], { ttl: '30d' })
await ai.forget(userId, 'VIP customer')                // one fact: true if it existed
await ai.forget(userId)                                // every fact
await bot.deleteSubject(userId)                        // right to erasure: memory included
```

| Method | Description |
| - | - |
| `memory(userId)` | Live facts, oldest first (`string[]`). |
| `memoryFacts(userId)` | The same with `createdAt`, `updatedAt`, `expiresAt` and `source` (`'auto'` or `'manual'`). |
| `remember(userId, facts, { ttl?, source? })` | Store one or more facts. Duplicates (case and punctuation ignored) renew the existing fact. |
| `forget(userId, fact?)` | Forget one fact, or everything without `fact`. |
| `extractMemory(userId, userText, assistantText)` | Run one extraction by hand; resolves with the facts added. |
| `idle()` | Resolves once background extraction has settled (tests, graceful shutdown). |

## RAG

`rag` accepts any **retriever**, an object with
`retrieve(query, { topK?, minScore?, signal? })` that resolves passages
`{ text, source?, citation?, score? }`. The
[`@botmesh/rag`](#@botmesh/rag) package below provides one with loaders,
chunking, embeddings and an index; a vector database or a search API works
just as well.

```ts theme={null}
import { aiPlugin } from '@botmesh/ai'
import { createRag } from '@botmesh/rag'

bot.plug(aiPlugin({
  provider: 'openai',
  model: 'gpt-5-mini',
  rag: {
    retriever: createRag({ documents: ['./docs/faq.md', './docs/products.pdf'], embeddings: 'openai' }),
    appendSources: true, // adds "Sources: [1] docs/faq.md › Shipping" to the reply
  },
}))
// User: "how long does shipping take?"
// Bot:  "It takes 3 business days [1].\n\nSources: [1] docs/faq.md › Shipping"
```

On every turn the plugin retrieves passages for the user's message and
injects them into the system prompt as a numbered block (`[1] (label) text`),
with instructions to use them when relevant, cite them as `[n]`, say it does
not know when the context has no answer, and treat the context as data, not
instructions.

| Field | Default | Description |
| - | - | - |
| `retriever` | — | The retriever (required in the object form). |
| `topK` | the retriever's default | Passages per question. |
| `minScore` | the retriever's default | Minimum score. |
| `appendSources` | `false` | `true` appends `"\n\nSources: [2] b"` with the passages the model cited, keeping the `[n]` it used in the text (several passages with the same label share one entry: `[1][3] faq.md`); when it cited none, every retrieved source, unnumbered (`Sources: a, b`). A function `(citations, cited) => string` renders the line yourself: `cited` holds `{ n, chunk }` for each cited passage. |
| `instructions` | built-in | Replaces the instructions that precede the context block. |
| `timeoutMs` | `10000` | Max wait for retrieval, including a first-time index build. Past it the retriever is aborted and the reply goes on without context. `0` disables. |

`rag: retriever` (the bare retriever) is the same as `rag: { retriever }`. The
default `appendSources` line is the localized `ai.rag.sources` message
(`Sources: {sources}`, or `Fuentes: {sources}` when `ctx.lang` is `es`);
override it with `bot.locale()` or pass a function. At install the plugin calls the retriever's optional `warmup()`
(non-fatal: a failure is logged), so the index is built at startup instead of
on the first question.

### Citations and facts of a turn

`replyInChatDetailed()` returns what the turn used:

```ts theme={null}
import { chatKeyOf } from '@botmesh/ai'

bot.when('help', async (ctx) => {
  const r = await ctx.tools.ai.replyInChatDetailed(chatKeyOf(ctx), ctx.body, {
    userId: ctx.sender,
    isGroup: ctx.isGroup,
  })
  r.text      // reply (with the sources appended when appendSources is on)
  r.answer    // the model's raw answer, as stored in the history
  r.citations // passages retrieved for this turn
  r.facts     // user facts injected into this turn
  await ctx.reply(r.text)
})
```

`replyInChat(key, text, { userId, isGroup })` returns only the text. Pass
`userId` to enable memory for the turn, and `isGroup` so facts stay out of
group replies.

With [streaming](https://botmesh.afaya.com.co/guide/streaming.md#with-@botmesh/ai), memory and RAG apply the
same way; with `appendSources` the sources line arrives as the last piece.

### `@botmesh/rag`

```bash theme={null}
pnpm add @botmesh/rag
```

It does not
depend on `@botmesh/ai`: `aiPlugin` takes any retriever and `createRag()` returns one.

`createRag(options)` builds a retriever without a vector database: an
in-memory cosine index with optional JSON persistence.

* **Loaders**: Markdown (split by heading sections), text, JSON (one document
  per array element) and PDF (with the optional `pdf-parse` peer, or your own
  `pdf` parser). Directories are read recursively.
* **Chunking**: paragraphs, then sentences, then words, with overlap.
* **Embeddings**: `'local'` (default; deterministic hashing, offline, lexical
  only, good for tests and small FAQs), `'openai'` (any OpenAI-compatible
  `/embeddings` endpoint, `text-embedding-3-small` and `OPENAI_API_KEY` by
  default), `'ollama'` (`nomic-embed-text` on `http://localhost:11434` by
  default), a `{ provider, ... }` object, or your own `Embedder`
  (`{ id, embed(texts) }`). Requests go through `secureFetch` from
  `@botmesh/http`.
* **Index**: `topK`, `minScore` and filters. With `persistPath` it is saved as
  JSON after each change and, on restart, only the chunks whose text changed
  are embedded again. If you switch to another embedder (a different `id`),
  the documents added at runtime and a prebuilt index are embedded again with
  it instead of being dropped; `onError` gets a `RagError('EMBEDDER_CHANGED')`
  and the old file is kept as `<persistPath>.previous`. If the index cannot be
  saved while it loads (read-only file), `onError` gets `PERSIST_FAILED` and
  it is served from memory.

| Option | Default | Description |
| - | - | - |
| `documents` | — | Paths (files or directories) and/or `{ id, text, source }` documents. Loaded lazily on first use. |
| `embeddings` | `'local'` | See above. |
| `chunkSize` | `1000` | Target characters per chunk. |
| `chunkOverlap` | `150` (at most `chunkSize / 4`) | Characters carried over from the previous chunk. |
| `topK` | `3` | Chunks returned by `retrieve()`. |
| `minScore` | `0` | Minimum cosine score. |
| `persistPath` | — | JSON file for the index. |
| `cwd` | `process.cwd()` | Base directory for relative paths. |
| `maxFileBytes` | 20 MiB | Max bytes per loaded file. |
| `retryDelayMs` | `30000` | After a failed ingest, `retrieve()` rejects at once with the same error for this long. |
| `onError` | — | Non-fatal problems, always a `RagError`: `INVALID_INDEX` (corrupt index set aside and rebuilt), `EMBEDDER_CHANGED` (re-embedded with another embedder), `PERSIST_FAILED` (could not save while loading; served from memory). |

`createRag()` is synchronous and cheap, so it fits in a synchronous plugin
install. You can also use it directly:

```ts theme={null}
import { createRag, formatContext } from '@botmesh/rag'

const rag = createRag({ documents: ['./docs'], embeddings: 'ollama' })

const hits = await rag.retrieve('payment methods', { topK: 2 })
// [{ text, score, source, section, citation: 'docs/faq.md › Payments', id, docId, index }]
const answer = await ctx.tools.ai.ask(`Context:\n${formatContext(hits)}\n\nQuestion: ${ctx.body}`)

await rag.addText('We open on Sundays from 10 to 14.', { source: 'hours' })
await rag.addFiles(['./docs/new'])
await rag.removeDocument('hours')
await rag.save()
```

Every failure is a `RagError` with a `code`: `INVALID_OPTIONS`,
`UNSUPPORTED_FORMAT`, `LOAD_FAILED`, `MISSING_DEPENDENCY`, `EMBEDDING_FAILED`,
`INVALID_INDEX` or `PERSIST_FAILED` (`EMBEDDER_CHANGED` only reaches
`onError`). The index is brute force, meant for thousands of chunks;
for millions, write your own retriever over a vector database. Details in the
[`@botmesh/rag` README](https://github.com/andresayac/botmesh/tree/main/packages/rag).

## Testing

`MockAIProvider` needs no network. To test memory, give the `AIClient` a
`MemoryManager` over the bot's profiles; to test RAG, any object with
`retrieve()` will do:

```ts theme={null}
import { AIClient, ContextManager, MemoryManager } from '@botmesh/ai'
import { MockAIProvider } from '@botmesh/ai/testing'
import { createTestBot } from '@botmesh/core/testing'

const { bot } = await createTestBot()
const mock = new MockAIProvider({ reply: 'It takes 3 days [1].' })
const ai = new AIClient(mock, new ContextManager(bot.getStore()), {
  memory: new MemoryManager((id) => bot.user(id)),
  rag: { retriever: { retrieve: async () => [{ text: 'Shipping takes 3 days.', citation: 'faq' }] }, appendSources: true },
})

await ai.remember('user-1', 'Prefers Spanish')
const r = await ai.replyInChatDetailed('chat-1', 'shipping?', { userId: 'user-1' })
r.text  // 'It takes 3 days [1].\n\nSources: [1] faq'
r.facts // ['Prefers Spanish']
mock.calls[0]?.system // contains the facts and the numbered context
```

## Next steps

* [AI plugin](https://botmesh.afaya.com.co/guide/ai.md) — providers and the full `aiPlugin` reference
* [AI tool calling](https://botmesh.afaya.com.co/guide/ai-tools.md) — let the model call your bot's functions
* [Persistent user data](https://botmesh.afaya.com.co/guide/user-data.md) — the `ctx.user` profile memory lives in


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.