> ## Documentation Index
> Fetch the complete documentation index at: https://prismeai-docs-next.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.
# Configuring Knowledges
> Set up the Knowledges product for RAG (Retrieval-Augmented Generation) with model orchestration, rate limits, and vector stores.
# Configuring Knowledges
**Knowledges** is Prisme.ai’s product for agentic assistants powered by tools and **retrieval-augmented generation (RAG)**. It enables teams to build agents that leverage internal knowledge across various formats, interact with APIs via tools, and collaborate with other agents through context sharing — enabling true multi-agent workflows with robust LLM support and enterprise-grade configuration options.
This guide explains how to configure Knowledges in a **self-hosted** environment.
***
## Core Capabilities
* Configure multi-model support with failover and fine-tuned prompts
* Automate agent provisioning via Builder
* Enforce **limits**, **security**, and **monitoring**
* Enable **builtin tools** like summarization, search, code interpreter, web browsing
* Integrate with **OpenSearch**, **Redis**, or other vector stores
***
## LLM Providers
Configure the `llm.openai.openai.models` field :
```yaml theme={null}
llm:
openai:
...
openai:
api_key: '{{secret.openaiApiKey}}'
models:
- gpt-4
- gpt-4o
- o1-preview
- o1-mini
```
Configure the `llm.openai.azure.resources.*.deployments` field.\
Multiple resources can be added by appending additional entries to the `llm.openai.azure.resources` array :
```yaml theme={null}
llm:
openai:
azure:
resources:
- resource: "resource name"
api_key: '{{secret.azureOpenaiApiKey}}'
api_version: '2023-05-15'
deployments:
- gpt-4
- embedding-ada
```
Configure the `llm.bedrock.*.models` and `llm.bedrock.*.region` fields.\
Multiple regions can be used by appending additional entries to the `llm.bedrock` array :
```yaml theme={null}
llm:
...
bedrock:
- credentials:
aws_access_key_id: '{{secret.awsBedrockAccessKey}}'
aws_secret_access_key: '{{secret.awsBedrockSecretAccessKey}}'
models:
- mistral.mistral-large-2402-v1:0
- amazon.titan-embed-image-v1
region: eu-west-3
- credentials:
aws_access_key_id: '{{secret.awsBedrockAccessKey}}'
aws_secret_access_key: '{{secret.awsBedrockSecretAccessKey}}'
models:
- amazon.titan-embed-text-v1
region: us-east-1
```
Configure the `llm.openai.vertex` field :
```yaml theme={null}
llm:
vertex:
credentials:
service_account: '{{secret.vertexServiceAccount}}'
host: us-central1-aiplatform.googleapis.com
models:
- projects/my-project-id/locations/us-central1/publishers/google/models/gemini-2.5-flash-preview-05-20
```
While deploying a model through Vertex the name of the model should represent the full endpoint name as in the above example.
The `modelAliases` feature comes really handy for this provider! In order to provide better readability to your users the above can be transformed into:
```yaml theme={null}
llm:
vertex:
credentials:
service_account: '{{secret.vertexServiceAccount}}'
host: us-central1-aiplatform.googleapis.com
models:
- vertex-gemini-2.5-flash-preview-05-20
...
modelAliases:
vertex-gemini-2.5-flash-preview-05-20: projects/my-project-id/locations/us-central1/publishers/google/models/gemini-2.5-flash-preview-05-20
```
Note that the `service_account` credentials should be ommitted if you deployed your platform on GCP and rely on IAM authentication. Also, the `service_account` value should either be :
* JSON
* Stringified JSON (handy if you save it within a secret)
To configure third-party models on Vertex (for example, Claude), please specify the format tag.
```yaml theme={null}
vertex:
- credentials:
service_account: '{{secret.vertexServiceAccount}}'
host: us-east5-aiplatform.googleapis.com
models:
- gemini-2.0-flash-us-east5
- imagen-3.0-generate-002-us-east5
- gemini-embedding-exp-03-07-us-east5
- credentials:
service_account: '{{secret.vertexServiceAccount}}'
host: us-east5-aiplatform.googleapis.com
format: anthropic
models:
- vertex-claude-sonnet-3.5-v2-20241022
```
Configure the `llm.openailike` field :
```yaml theme={null}
llm:
...
openailike:
- api_key: "{{config.apiKey1}}"
endpoint: "endpoint 1"
models:
- mistal-large
- api_key: "{{secret.apiKey2}}"
endpoint: "endpoint 2"
provider: Mistral
models:
- mistral-small
- mistral-mini
options:
excludeParameters:
- presence_penalty
- frequency_penalty
- seed
headers:
Custom-Header: my-static-value
User-Custom-Header: '{userId}'
Another-Custom-Correlation: '{correlationId}'
```
**Optional Parameters:**
* **provider**: The provider name used in analytics metrics and dashboards.
* **options.excludeParameters**: Allows exclusion of certain OpenAI generic parameters not supported by the given model.
* **options.headers**: Custom HTTP headers sent with every request to this provider. Values can be static strings or use `{correlationId}` / `{userId}` as the full value to inject runtime context.
**Gemini integration :**
```yaml theme={null}
- api_key: '{{secret.geminiApiKey}}'
endpoint: https://generativelanguage.googleapis.com/v1beta/openai/
models:
- gemini-2.0-flash
- gemini-2.0-flash-lite
- gemini-1.5-pro
- gemini-2.5-pro-preview-03-25
options:
excludeParameters:
- presence_penalty
- frequency_penalty
```
***
## Global Configuration
```yaml theme={null}
defaultModels:
completions: gpt-4o
embeddings: az-embedding-ada
enhanceQuery: gpt-3.5-turbo
toolRouting: gpt-4o
```
Set base models for completions, embeddings, and query enhancement.
Configure the following fields at the root of the configuration :
```yaml theme={null}
openai:
model-embeddings: '{{config.defaultModels.embeddings}}' # The default model to use for embeddings
model: '{{config.defaultModels.completions}}' # The default model to use for LLM completions
temperature: 0.1
max_tokens: 2000 # Max LLM response size, INCLUDING reasoning
top_p: 1
frequency_penalty: 0
presence_penalty: 0.6
textSplitter:
chunkSize: 1000
chunkOverlap: 200
indexInitialized: true
embeddings:
numberOfSearchResults: 10
maxContext: 2048
```
Rate limits can currently be applied at two stages in messages processing :
1. When a message is received (**requestsPerMinute** limits for projects or users).
2. After RAG stages and before the LLM call (**tokensPerMinute** limits for projects, users, models, or **requestsPerMinute** limits for models).
Embedding model rate limits are applied before vectorizing a document, per project or model.
This is how to configure token and request limits globally or per user/project:
```yaml theme={null}
limits:
files_count: 20
llm:
users:
tokensPerMinute: 100000
requestsPerMinute: 20
projects:
tokensPerMinute: 30000
requestsPerMinute: 300
embeddings:
projects:
tokensPerMinute: 1000000
```
* **limits.llm.users**: Defines per-user message/token limits across all projects.
* **limits.llm.projects**: Defines default message/token limits per project. **These limits can be overridden per project via the /admin page in Knowledges.**
* **limits.files\_count**: Specifies the maximum number of documents allowed in Knowledges projects. **This number can also be overridden per project via the /admin page.**
See [Models specifications](#models-configuration) for rate limits per model.
If you have multiple LLM Providers or regions with the same model names (for example gpt-4), you can use custom names:
```yaml theme={null}
llm:
openai:
azure:
resources:
- resource: "resource name"
api_key: '{{secret.azureOpenaiApiKey}}'
api_version: '2023-05-15'
deployments:
- gpt-4-openai
openai:
api_key: '{{secret.openaiApiKey}}'
models:
- gpt-4-azure
```
And you can map them to the name expected by the provider with the following:
```yaml theme={null}
modelAliases:
gpt-4-openai: gpt-4
gpt-4-azure: gpt-4
```
As a reminder, here is how modelsSpecifications could look like :
```yaml theme={null}
modelsSpecifications:
gpt-4-openai:
displayName: GPT 4 OpenAi
maxContext: 8192
...
gpt-4-azure:
displayName: GPT 4 Azure
maxContext: 8192
...
```
If you have your own SSO configured, you need to explicitly allow SSO authenticated users to access Knowledges pages :
1. Open **Knowledges** workspace
2. Open **Settings** > **Advanced**
3. **Manage roles**
4. Add your SSO provider technical name after `prismeai: {}` at the very beginning :
```yaml theme={null}
authorizations:
roles:
editor: {}
free:
auth:
prismeai: {}
yourOwnSso: {}
```
By default, sharing an agent with an external email will automatically send an invitation mail to let the external user create an account and access the agent.
You can disable this to enforce user control :
```yaml theme={null}
disableAccountCreation: true
```
Only existing users will be able to access shared agents.
Knowledges supports onboarding flows, multilingual statuses, and customizable notifications:
```yaml theme={null}
status:
colors:
published: '#5CA44A'
pending: '#FF9261'
draft: '#E5E5E5'
prompt:
default: |
You will only answer based on this context:
${context}
toasts:
i18n:
fr:
documentCrawled: a été indexé par votre IA 🤖
en:
documentCrawled: was indexed by your AI 🤖
```
***
## Models Configuration
Configure all available models with descriptions, rate limits, and failover:
```yaml theme={null}
modelsSpecifications:
gpt-4o:
displayName: GPT-4o
maxContext: 128000
maxResponseTokens: 2000
isHiddenFromEndUser: true
subtitle:
fr: Modèle hébergé aux USA.
en: Model hosted in the USA.
description:
fr: Le modèle GPT-4o sur OpenAI. Vous pouvez utiliser des documents C1 et C2.
en: The GPT-4o model on OpenAI. You can use documents C1 and C2.
rateLimits:
requestsPerMinute: 1000
tokensPerMinute: 100000
failoverModel: 'gpt-4o'
region: eu-west
environmentalMetrics:
energyPerToken: 4.35e-7
pueProfile: efficient
display:
brand: Open AI
name: GPT-4o
icon: https://staging-uploads.prisme.ai/cAZmE4C/r3G28CGTInbQSkVXXfDw4.open-ai.svg
ecoScore: high
trainingDate: '2024-11-20T00:00:00.000Z'
cost: low
capabilities:
text:
enabled: true
vision:
enabled: true
image:
enabled: true
file:
enabled: true
maxSize: 20000000
text-embedding-ada-002:
type: embeddings
maxContext: 2048
batchSize: 96
subtitle: {}
description: {}
mistral.mistral-large-2402-v1:0:
maxContext: 120000
additionalRequestBody:
completions:
guardrailConfig:
guardrailIdentifier: "..."
guardrailVersion: '1'
embeddings: {}
some-bedrock-model:
maxContext: 120000
promptCache:
system: true
```
* All LLM models (excluding those with `type: embeddings`) will automatically appear in the Agent Creator menu unless disabled at the agent level, with the configured titles and descriptions.
* `displayName` specifies the user-facing name of the model, replacing the technical or original model name to ensure a more intuitive and user-friendly experience.
* `isHiddenFromEndUser` specifies that a model in the configuration will be hidden from end users. This feature allows administrators to temporarily disable a model or conceal it from the end-user interface without permanently removing it from the configuration.
* `maxContext` specifies the maximum token size of the context that can be passed to the model, applicable to both LLMs (full prompt size) and embedding models (maximum chunk size for vectorization).
* `maxResponseTokens` defines the maximum completion size requested from the LLM, which can be overridden in individual agent settings.
* `additionalRequestBody.completions` and `additionalRequestBody.embeddings` specify custom parameters which will be sent within all HTTP request bodies for the given model, used to enable AWS Guardrails in above example
By default, documents paragraphs are vectorized in batches of 96.\
You can customize this `batchSize` per model :
```yaml theme={null}
modelsSpecifications:
text-embedding-ada-002:
type: embeddings
maxContext: 2048
batchSize: 50
```
Or globally :
```yaml theme={null}
embeddings:
batchSize: 50
```
When `modelsSpecifications.*.rateLimits.requestsPerMinute` or `modelsSpecifications.*.rateLimits.tokensPerMinute` are defined, an error (customizable via `toasts.i18n.*.rateLimit`) is returned to any user attempting to exceed the configured limits. These limits are shared across all projects/users using the models.
If these limits are reached and `modelsSpecifications.*.failoverModel` is specified, projects with `failover.enabled` activated (disabled by default) will automatically switch to the failover model.
**Notes:**
* **tokensPerMinute** limits apply to the entire prompt sent to the LLM, including the user question, system prompt, project prompt, and RAG context.
* Failover and **tokensPerMinute** limits also apply to intermediate queries during response construction (e.g., question suggestions, self-query, enhanced query, source filtering).
Environmental metric can be calculated when using models by setting the region where the model is hosted :
```
region: world | eu-west | eu-north | us-east | us-west | asia
```
energy consumed per token (in kWh) and PUE (Power Usage Effectiveness) profile :
```
environmentalMetrics:
energyPerToken: 4.35e-7
pueProfile: efficient | average | inefficient
```
The display section defines the model’s brand, name, icon, eco-score, and cost information:
```yaml theme={null}
display:
brand: Open AI
name: GPT-4o
icon: [https://staging-uploads.prisme.ai/cAZmE4C/r3G28CGTInbQSkVXXfDw4.open-ai.svg](https://staging-uploads.prisme.ai/cAZmE4C/r3G28CGTInbQSkVXXfDw4.open-ai.svg)
ecoScore: high
trainingDate: '2024-11-20T00:00:00.000Z'
cost: low
```
The capabilities section lists which modalities the model supports:
```yaml theme={null}
capabilities:
text:
enabled: true
vision:
enabled: true
image:
enabled: true
file:
enabled: true
maxSize: 20000000
```
**Notes:**
* Models from **OpenAI**, **Gemini**, and **Bedrock** currently support the **file** capability.
* **Azure OpenAI** does *not yet* provide this feature, although a **community feature request** is in progress.
* When `file.enabled` is set to `true` and the file size is within supported limits, the file is sent **directly in Base64** to the LLM.
* Upcoming releases will introduce support for passing a **file ID** instead of raw Base64 data, using the `storageProvider` parameter (`prisme`, `gcs`, or `s3`). This will enable seamless handling of **larger documents** by referencing files stored in connected cloud storage rather than embedding their content directly.
* the `file.maxSize` parameter is in Octet.
Some Bedrock models support [static prompt caching](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html#prompt-caching-simplified).\
You can enable this with `promptCache.system` option :
```yaml theme={null}
promptCache:
system: true
```
***
## Vector Store Configuration
To enable retrieval-based answers, configure a vector store:
```yaml theme={null}
vectorStore:
provider: redisSearch
url: '{{secret.redisUrl}}'
vectorIndexPrefix: 'aik_rag_'
```
Or with OpenSearch:
```yaml theme={null}
vectorStore:
provider: openSearch
url: '{{secret.opensearchUrl}}'
user: '{{secret.opensearchUser}}'
password: '{{secret.opensearchPassword}}'
vectorIndexPrefix: 'aik_rag_'
```
Or with ElasticSearch:
```yaml theme={null}
vectorStore:
provider: elasticSearch
url: '{{secret.elasticUrl}}'
user: '{{secret.elasticUser}}'
password: '{{secret.elasticPassword}}'
vectorIndexPrefix: 'aik_rag_'
```
***
## Tools and Capabilities
Knowledges enables advanced agents via tools.
RAG tool for semantic search within indexed documents.
Summarize entire files when explicitly requested.
Used to extract context from project knowledge collections.
Optional tool enabled via Serper API key:
```yaml theme={null}
tools:
webSearch:
apiKey: '{{secret.serperApiKey}}'
```
Python tool for data manipulation and document-based computation.
Uses DALL-E or equivalant if enabled in LLM config.
***
## Advanced Features
Knowledges projects and agents can be provisioned programmatically via Builder workflows.
Specify a backup model to switch to if the main one is overloaded:
```yaml theme={null}
failoverModel: gpt-3.5-turbo
```
Make sure to enable failover in your workspace.
Assign costs per million tokens to track model usage:
```yaml theme={null}
pricing:
input: 2.5
output: 10
```
This can be used with usage-based dashboards in Insights.
\--
## Operations - ES/OS
Knowledges creates between 1 and 2 indices per agent (so 2 - 4 shards with default redundacy settings).
### 1. Crawler Storage
The **crawler** stores the full text content of crawled documents. This includes the raw extracted text from web pages, PDFs, and other document types that have been processed.
### 2. Vector Store
The **vector store** (when using ES/OS as your vector database) stores the embeddings for each paragraph/chunk extracted from documents. These vectors enable semantic search and RAG retrieval.
### Index Structure
These two types of data are stored in **separate indexes**:
| Index | Purpose | Content | Index prefix |
| ------------- | ----------------- | ----------------- | ---------------------------------------- |
| Crawler index | Full-text storage | Raw document text | `-searchengine-webpages-` |
| Vector index | Semantic search | Chunk embeddings | `__` |
* **vectorIndexPrefix** is the prefix configured in Knowledges configuration at `vectorStore.vectorIndexPrefix` key
* **AIK\_ID** is the Knowledges workspace id
* **crawlerPrefix** is the `ELASTIC_INDICES_PREFIX` environment variable configured in `prismeai-searchengine` and `prismeai-crawler` services
### Cleanup
Indexes are created automatically when the **first document** is added to an Knowledges project.
As documents are deleted over time, these indexes may eventually contain **zero documents**—this can happen for both the crawler index and the vector store index.
**Important:** Empty indexes still consume shards. To free up resources, it is recommended to delete empty indexes either:
* Manually via **Kibana** / **OpenSearch Dashboards**
* Through an **automated cleanup task**
This helps maintain optimal cluster performance and avoid hitting shard limits.
***
## Next Steps
Create a ChatGPT-like agent within your organization — but secure, customizable, and connected to your tools and knowledge.
Monitor usage and LLM activity
Learn more about agent creation