> ## Documentation Index
> Fetch the complete documentation index at: https://prismeai-docs-next.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Website to RAG Agent

> Create a transformative Gen.AI chatbot for your website using RAG architecture

This tutorial guides you through creating an intelligent chatbot for your website using Retrieval-Augmented Generation (RAG) architecture. Your chatbot will automatically read and understand your website content, allowing visitors to ask questions and receive accurate, contextual responses—similar to interacting with ChatGPT but specifically trained on your website's information.

## What You'll Build

A complete website chatbot with:

* Automatic content crawling and indexing
* Conversational AI powered by RAG architecture
* Customizable chat interface
* Easy website integration with a simple script
* Analytics and performance monitoring

<Note>
  This solution transforms static website content into an interactive knowledge base, enhancing user engagement and providing a powerful way for visitors to find information instantly.
</Note>

## Prerequisites

Before starting this tutorial, make sure you have:

* An active Prisme.ai account
* A website with content you want to make interactive
* Basic understanding of HTML (for embedding the chat widget)

## Step 1: Creating Your Website RAG Workspace

Let's start by setting up a dedicated workspace using a pre-built template:

<Steps>
  <Step title="Access Prisme.ai Studio">
    Log in to your Prisme.ai account and navigate to the Studio builder
  </Step>

  <Step title="Find the Website to RAG Template">
    Look for the "Website to RAG" option in the project suggestions
  </Step>

  <Step title="Duplicate the Template">
    Click on "duplicate" to create your workspace based on this template
  </Step>

  <Step title="Explore the Activity Feed">
    You'll be automatically directed to the activity feed, which logs all actions within your workspace
  </Step>
</Steps>

## Step 2: Getting Familiar with the Workspace

Now, let's explore the components of your new workspace:

<Steps>
  <Step title="Expand the Sidebar">
    Click on the menu icon to expand the sidebar, revealing Pages and Automations
  </Step>

  <Step title="View Installed Apps">
    Navigate to the Apps section to see pre-installed applications
  </Step>

  <Step title="Identify Key Components">
    Note the main components:

    * **Crawler App**: For fetching and processing website content (based on robots.txt rules)
    * **Knowledge Client**: For linking your workspace to Knowledges Agent based on Agent Id & API Key.
  </Step>
</Steps>

## Step 3: Configuring the Website Crawler

The crawler is the heart of your RAG agent, responsible for reading and processing your website's content:

<Steps>
  <Step title="Access Crawler Configuration">
    In the Apps section, click on the "Crawler" app
  </Step>

  <Step title="Enter Your Website URL">
    Input the URL of the website you want the chatbot to read
  </Step>

  <Step title="Configure Crawling Parameters">
    Adjust settings as needed:

    * **Crawl Depth**: How many levels of links to follow
    * **URLs to Include or Exclude**: Specific patterns to focus on or skip
    * **Authentication**: Set up authentication if required.
    * **Periodicity**: Control the periodicity of URLs wrawling
  </Step>

  <Step title="Save Configuration">
    Click the "Save" button to apply your crawler settings
  </Step>

  <Step title="Monitor Crawling Progress">
    Navigate to the "Crawler" page under "Pages" to view the current status
  </Step>
</Steps>

<Note>
  The crawler may take some time to process your website, depending on its size. You can begin testing with partially crawled content while the process continues in the background. A maximum document quota is often enforced on your instance. Please contact your internal point of contact to request an increase of this limit.
</Note>

## Step 4: Testing Your RAG Chatbot

Once the crawler has processed some of your website content, you can start interacting with your chatbot:

<Steps>
  <Step title="Open the Chat Page">
    Navigate to the Chat page in your workspace
  </Step>

  <Step title="Ask Questions About Your Website">
    Try asking questions related to the content on your website
  </Step>

  <Step title="Explore Available Documents">
    If you're unsure what to ask, explore the AI knowledge hub from the sidebar
  </Step>

  <Step title="Test with Specific Queries">
    Select a document from the knowledge hub and formulate questions based on its content
  </Step>
</Steps>

## Step 5: Customizing the Chat Widget

Now let's customize how your chat interface will appear on your website:

<Steps>
  <Step title="Open Agent Creator">
    Navigate to **your agent**, click on **“Share Agent”**, and copy the page URL or embed script.
  </Step>

  <Step title="Customize Appearance">
    In the script, you can adjust the widget settings to match your website's design:

    * **Background Color**: Define the chat window’s background color
    * **Chat Icon**: Choose the icon that visitors click to open the chat
    * **Close Icon Color**: Set the color of the close (X) button
  </Step>

  <Step title="Advanced Styling">
    Use the CSS editor for deeper customization and fine-grained control of the widget’s style.
  </Step>

  <Step title="Preview Changes">
    Instantly preview how your changes affect the widget’s appearance in real time.
  </Step>
</Steps>

## Step 6: Integrating the Chatbot into Your Website

Now it's time to add your chatbot to your website:

<Steps>
  <Step title="Get the Embed Code">
    On the Widget page, locate and copy the provided script code
  </Step>

  <Step title="Add the Script to Your Website">
    Paste the script code into your website's HTML, ideally just before the closing `</body>` tag

    ```html theme={null}
    <script src="https://cdn-assets.prisme.ai/blocks/popover/inject/index.js"></script>
    <script type="text/javascript">
    Prismeai.popover({
    "url": "https://ai-knowledge-chat.pages.prisme.ai/agent?id=your-agent-id",
    "config": {
    "header": {
      "closeColor": "#7f7f7f",
      "title":" "
    },
    }
    })
    </script>

    ```
  </Step>

  <Step title="Test Integration">
    Visit your website to verify that the chat widget appears and functions correctly
  </Step>
</Steps>

## Step 7: Understanding the RAG Architecture

Your website chatbot is powered by a sophisticated RAG (Retrieval-Augmented Generation) architecture:

<Frame>
  <img src="https://mintcdn.com/prismeai-docs-next/TwMZYLd6Xg1G75e6/images/rag-architecture.png?fit=max&auto=format&n=TwMZYLd6Xg1G75e6&q=85&s=4d58fe866a9fedf1d1067f655fb25a42" alt="RAG Architecture" width="9220" height="5396" data-path="images/rag-architecture.png" />
</Frame>

The system works through several key processes:

1. **Document Processing**: Your website content is crawled, segmented into chunks, and converted into vector embeddings
2. **Query Understanding**: When a user asks a question, the system processes and enhances it for better retrieval
3. **Relevant Content Retrieval**: The system searches for the most relevant chunks of your website content
4. **Response Generation**: Using the retrieved content as context, the AI generates a comprehensive, accurate response

<Note>
  Knowledges breaks documents into optimally sized "chunks" for efficient retrieval. This chunking process is configurable, allowing you to adjust settings like chunk size and overlap to match your specific content needs.
</Note>

## Step 8: Advanced Customization

For developers looking to extend the functionality:

<Steps>
  <Step title="Access Advanced Settings">
    In Knowledges, you can fine-tune embeddings, chunk management, and query enhancement
  </Step>

  <Step title="Use the Webhook Feature">
    Implement custom RAG pipelines using the Webhook feature
  </Step>

  <Step title="Add Python Custom Code">
    For advanced implementations using LlamaIndex or Langchain, use Python Custom Code capabilities within the Builder
  </Step>

  <Step title="Explore Templates">
    Check the suggestions section of the Low-code Builder product for predefined templates designed for specific RAG pipelines
  </Step>
</Steps>

## Step 9: Monitoring and Optimization

Keep your website chatbot performing at its best:

<Steps>
  <Step title="Monitor Usage">
    Track user interactions and popular questions to understand how visitors use your chatbot
  </Step>

  <Step title="Review Performance">
    Analyze response times and accuracy to identify areas for improvement
  </Step>

  <Step title="Update Content">
    Regularly recrawl your website to ensure the chatbot has access to the latest content
  </Step>

  <Step title="Refine Responses">
    Adjust settings based on user feedback to improve response quality
  </Step>
</Steps>

## Best Practices for Website RAG Agents

To maximize the effectiveness of your website chatbot:

<CardGroup cols="2">
  <Card title="Content Structure" icon="sitemap">
    Organize your website content with clear headings and logical structure for better chunking
  </Card>

  <Card title="Regular Updates" icon="arrows-rotate">
    Schedule regular recrawling to keep your chatbot's knowledge current
  </Card>

  <Card title="User Guidance" icon="hand-pointer">
    Provide example questions to help users understand what they can ask
  </Card>

  <Card title="Response Tuning" icon="sliders">
    Adjust chunk size and retrieval settings based on the nature of your content
  </Card>
</CardGroup>

## Conclusion

By following this tutorial, you've successfully transformed your static website into an interactive knowledge base with an intelligent chatbot. Your visitors can now engage with your content in a natural, conversational way, finding information faster and more intuitively.

This implementation not only enhances user experience but also provides valuable insights into what information your visitors are seeking, helping you continually improve your website content and structure.

## Next Steps

<CardGroup cols="2">
  <Card title="Build a Document Classification System" icon="file" href="/resources/tutorials/data-classification-agent">
    Create an AI system that automatically organizes uploaded documents
  </Card>

  <Card title="Create a No-Code RAG Agent" icon="robot" href="/resources/tutorials/no-code-rag-agent">
    Explore more advanced RAG capabilities without writing code
  </Card>

  <Card title="Implement Webhook Integration" icon="webhook" href="/resources/tutorials/webhook-builder">
    Connect your RAG agent to other systems using webhooks
  </Card>

  <Card title="Develop Custom Applications" icon="puzzle-piece" href="/resources/tutorials/ai-builder-custom-apps">
    Build more sophisticated applications using Builder
  </Card>
</CardGroup>