How to restrict GPT API responses to a custom dataset

When interacting with the OpenAI API (also referred to as the GPT API), a common challenge is ensuring the model's responses are relevant and specific to a custom dataset. We will introduce several strategies to help custom-fit the OpenAI API's responses to a particular dataset.

Key takeaways:

  • Crafting targeted prompts can help the OpenAI API produce responses specific to a custom dataset.

  • Binary classifiers can pre-screen queries to ensure only relevant questions are processed, maintaining focus on your dataset.

  • Setting similarity thresholds helps verify that responses align closely with your dataset, improving relevance.

  • OpenAI’s moderation tools can filter out inappropriate or irrelevant content to uphold response quality.

Prompt engineering

Prompt engineering is a technique that involves crafting your prompts in a way that explicitly states the kind of questions that should not be answered. For instance, you could instruct the model to refrain from answering questions that are not related to a specific domain. This technique is all about setting clear boundaries for the AI model right from the start.

The following code allows you to experiment with different prompts and observe the outputs. Feel free to adjust the parameters and see how the model’s responses change. The code will need to input your API key to provide an output.

import openai
import os
openai.api_key = os.environ["SECRET_KEY"]
# Specific prompt to restrict responses to a custom dataset
prompt = (
    "You are an assistant that only provides information based on the following datas:\n"
    "Dataset: [Insert your dataset details here]\n\n"
    "If the question is related to the dataset, provide a concise and accurate answer.\n"
    "If the question is outside the scope of the dataset, respond with 'I can only provide information based on the specified dataset.'\n"
    "If the dataset isn't provided, simply respond with 'Dataset not provided yet, I can't answer questions out of scope'.\n"
    "Question: What is the process for data validation in our system?"
)
# Create a completion using the chat completions API
response = openai.chat.completions.create( # Use openai.chat.completions.create() for chat models
    model="gpt-4o",
    messages=[
        {"role": "user", "content": prompt}
    ],
    max_tokens=150
)
# Print the response
print(response.choices[0].message.content.strip())

The prompt includes clear instructions that the assistant should only use information from the specified dataset. With some careful prompt engineering, we can restrict the OpenAI API’s responses to specific contexts. If you want to learn more about the intricacies of prompt engineering, check out our “All You Need to Know About Prompt Engineering” course.

Binary classifiers

Binary classifiers can be utilized to determine whether a query is “on-topic” or “off-topic” for your specific application. You can employ fine-tuned OpenAI models or even open-source alternatives for this task. In this manner, you can confirm that the AI model only addresses queries relevant to your dataset.

Example

# Assuming 'classifier' is a trained binary classifier
is_on_topic = classifier.predict(question)
if is_on_topic:
  response = openai.chat.completions.create( # Use openai.chat.completions.create() for chat models
      model="gpt-4o",
      messages=[
          {"role": "user", "content": prompt}
      ],
      max_tokens=150
  )
Binary classifier

Similarity threshold

Implementing a minimum similarity threshold for retrieving documents to address questions ensures that the AI model’s responses closely align with your dataset. If no documents exceed this threshold, the model is guided to politely refrain from providing an answer.

Example

# Assuming 'similarity_score' is a function that calculates similarity
if similarity_score(question, document) > threshold:
  response = openai.chat.completions.create( # Use openai.chat.completions.create() for chat models
      model="gpt-4o",
      messages=[
          {"role": "user", "content": prompt}
      ],
      max_tokens=150
  )
else:
    response = "I'm sorry, I can't provide a relevant answer to your question."
Checking similarity threshold

Content moderation

Content moderation involves filtering out inappropriate requests. OpenAI offers a moderation endpoint for this purpose, which can be incredibly helpful in preserving the quality and relevance of the responses.

Example

Note: Replace your-openai-api-key with your API key.

import requests
import openai
openai.api_key = "your-openai-api-key"
def moderate(text):
    url = "https://api.openai.com/v1/moderations"
    headers = {
        "Authorization": f"Bearer your-openai-api-key",
        "Content-Type": "application/json"
    }
    data = {
        "prompt": {"content": text},
        "max_tokens": 60
    }
    response = requests.post(url, headers=headers, json=data)
    result = response.json()
    if 'choices' in result and len(result['choices']) > 0:
        return result['choices'][0]['finish_reason'] != 'stop'
    else:
        return False
def generate_response(question):
    if moderate(question):
        response = openai.Completion.create(
            engine="text-davinci-002",
            prompt=prompt,
            temperature=0.5,
            max_tokens=260
        )
    else:
        response = "I'm sorry, I can't process your request."
    return response

Code explanation

  • Lines 1–2: Imports the requests and openai libraries.

  • Line 4: Sets your OpenAI API key.

  • Lines 6–15: In the moderate function, configures and sends a POST request to OpenAI’s moderation endpoint with the text to be moderated.

  • Lines 16–22: The function checks the moderation response and returns True or False based on whether the text passed moderation.

  • Lines 24–33: In the generate_response function, if the text passes moderation, a text completion request is sent to OpenAI’s API using the text-davinci-002 engine with specified parameters. If the moderation fails, an error message is returned.

  • Line 34: Returns the generated response or the error message.

Word embeddings

Generating word embeddings from the data associated with your business concept or content and storing that information in a vector database enables you to search your database for context. If found, pass the same search to OpenAI’s ChatGPT, also transmitting the embeddings in your local database to the same prompt. This strategy gives you granular control over the provided responses.

Example

# Assuming 'word_embeddings' is a function that generates word embeddings
# and 'vector_db' is a vector database
embeddings = word_embeddings(question)
similar_contexts = vector_db.search(embeddings)
if similar_contexts:
    response = openai.Completion.create(
        engine="text-davinci-002",
        prompt=question,
        temperature=0.5,
        max_tokens=260
    )
else:
    response = "I'm sorry, I can't provide a relevant answer to your question."
Word embeddings

Conclusion

Limiting GPT API responses to a custom dataset may be a complex task, but it’s feasible with the right strategies. The techniques discussed in this Answer aren’t mutually exclusive and can be combined for optimal results. The objective is to ensure that the AI model supplies responses that are relevant, precise, and specific to your dataset.

Frequently asked questions

Haven’t found what you were looking for? Contact Us

Can binary classifiers effectively filter out irrelevant queries for the OpenAI API?

Yes, binary classifiers can be highly effective in filtering out irrelevant queries. By training a classifier to distinguish between “on-topic” and “off-topic” questions, you can pre-screen user inputs and ensure that only relevant queries are processed by the OpenAI API, thereby maintaining the quality and relevance of the responses.

How do similarity thresholds improve the relevance of OpenAI API responses?

Implementing similarity thresholds involves calculating the similarity between user queries and your custom dataset. By setting a minimum similarity score, you ensure that only queries closely related to your dataset are addressed by the API. If a query doesn’t meet the threshold, the model can be prompted to refrain from providing an answer, thereby enhancing response relevance.

What role do word embeddings play in controlling AI responses with the OpenAI API?

Word embeddings convert your custom dataset into numerical vectors that capture semantic meanings. By storing these embeddings in a vector database, you can perform efficient similarity searches to find relevant context for user queries. This ensures that the OpenAI API generates responses that are not only accurate but also deeply aligned with the specific content of your dataset.

Free AI Mock Interviews
Coding Interview
Coding PatternsFree Interview
Gain insights and practical experience with coding patterns through targeted MCQs and coding problems, designed to match and challenge your expertise level.
System Design
YouTubeFree Interview
Learn to design a video streaming platform like YouTube by tackling functional and non-functional requirements, core components, and high-level to detailed design challenges.

Free Resources

Copyright ©2025 Educative, Inc. All rights reserved