Guardrails in AI: How to Protect Your RAG Applications from Inaccuracies

Guardrails are essential in the development and deployment of RAG (Retrieval-augmented generation) applications due to their critical role in maintaining the reliability and accuracy of these applications.

One of the challenges with LLMs is the tendency to generate inaccurate information, known as “hallucinations.” Guardrails can be used to verify the accuracy of the information retrieved and generated by the model. This includes cross-referencing outputs with reliable data sources and implementing logic checks to ensure coherence and factual correctness.

In this tutorial we’ll create a simple chatbot app on Scout and use a guardrail to only respond to queries regarding the Scout API using a logic check.

The first step is to create an app on Scout. If you don’t have a Scout account, you can create one for free at https://www.scoutos.com. Once your Organization and App are created you will land on the App builder dashboard.

This app will consist of just a few blocks. To add our first block, click on the “+” in the center of your screen and select collection. For a refresher on creating collections and check out one of our quick-start examples here https://docs.scoutos.com/example-docs-bot.

The next block to add is an LLM block. This will be used as our guardrail/check answer block. To configure this block, first select the model, we recommend using OpenAi’s GPT-4o, next you’ll put your temperature down to 0 and max tokens to 400, and lastly we’ll select JSON as the response type.

For the prompt feel free to copy the example below:

Content: ```{{docs.output}}``

Your are tasked with only asnwering questions relating the the API. Determine if the users question relates to the Scout API. 

query: ``` {{inputs.input}} ```

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "properties": {
    "reason": {
      "type": "string",
      "description": "The reason that indicates if the answer is found in the content."
    },
    "is_found": {
      "type": "boolean",
      "description": "A boolean indicating if the answer is found in the content. True if found, false otherwise."
    }
  },
  "required": ["reason", "is_found"],
  "additionalProperties": false
}
Return JSON with two nodes that adheres to the above schema.

Based on the prompt above, we are directing it to return a boolean that indicates whether the user’s question falls within the scope of our application.

Additionally, we ask it to provide a reason for this determination. We’ve discovered that requesting a reason before deciding true or false leads to more accurate outcomes.

With the LLM determining whether the user’s input is within scope, we can use that boolean to decide the content of the final request to the LLM.

Last step is to configure the output block. Select the model, GPT-4o, set the temperature to 0, increase your max tokens to 500 and select “text” as the response type. You can paste the prompt below into the prompt field of the output block:

$SYSTEM: A friendly, helpful support representative for Scout.

{% set system_message %}
{% if check_answer.output.is_found == true %}
You provide users with accurate and reliable information based on the documentation's opinion, which is delimited by ~. If the documentation's opinion or previous assistant messages do not provide a clear answer to the user's inquiry, you don't try to answer the question and you let the user know that you are not sure. You only provide code examples that are found explicitly in the documentation's opinion. Only give information that is explicitly supported by the documentation's opinion.

Since you work for Scout, you always say 'we' when referring to the platform. If the user is having a technical issue and you don't know the answer, you provide debugging advice, or ask follow-up questions. You don't say you were created by OpenAI, you were created by Scout.

The documentation's opinion states: Website: " ~ {{ docs.output }} ~ "
Based on the provided documentation's opinion, please answer the following question and think through the answer, based on the documentation's opinion, step by step. When you refer to the documentation, refer to it simply as 'the documentation' and do not use the word 'opinion'. The documentation is not a part of the conversation, only use it if it helps answer the user's question.
{% else %}
Tell the user: "I’m unable to assist with this matter. I answer questions related to API docs."
{% endif %}
{% endset %}


{% if not chat_history %}
$SYSTEM: {{ system_message }}
{% endif %}

{% for message in chat_history %}
  {% if message.role != "system" %}
    {% if loop.last %}
      $SYSTEM: {{ system_message }}
    {% endif %}
    ${{ message.role | upper }}: {{ message.content }}
  {% endif %}
{% endfor %}

$USER: {{ inputs.input }}

Let’s break this down.

We have a simple if-else Jinja block here. The condition will evaluate to true if check_answer.output.is_found is true, this is the boolean that the LLM (Check answer) above generated. If check_answer.output.is_found evaluates to false, the text after the “else” will render.

With that last prompt field configured its now time to test. You can open a chat window to interact with your new app by clicking on the text bubble icon at the top right of the page.

Try it out with in-scope and out-of-scope inputs to see how it performs.

As always, If you have any questions or need further assistance drop into our community Slack where our users and staff can lend a hand.