buildfastwithaibuildfastwithai
AI WorkshopsAll blogsPartner with usFDE Program
Unrot mascotGet yourselfupskilled in AIfor free.Get the appNext live workshopFree AI WorkshopLive session, recording includedReserve a seat

Newsletter

Stay ahead

AI tools and tips. No spam.

Share
Back to blogs
Tools
Optimization

GPTCache: Supercharge Generative AI

By Satvik Paramkusham
February 17, 2025
5 min read
Share:
GPTCache: Supercharge Generative AI
Share:

Are you waiting for the future to happen or ready to make it happen?

Don’t miss your chance to join Gen AI Launch Pad 2025 and shape what’s next.

Introduction

With the increasing use of Generative AI models like GPT-4, developers and businesses face challenges related to latency, cost, and efficiency. GPTCache is a powerful caching library designed to optimize the performance of Large Language Model (LLM) applications by storing and reusing previous responses. This not only reduces redundant API calls but also enhances user experience with faster response times.

In this blog, we’ll explore the capabilities of GPTCache, break down the code required to integrate it into AI applications, and discuss best practices for maximizing efficiency. Whether you're working on chatbots, Retrieval-Augmented Generation (RAG) systems, or other AI-driven applications, this guide will help you unlock the full potential of GPTCache.

Setting Up GPTCache

Before integrating GPTCache into your AI workflow, you need to install the required dependencies. The following command installs GPTCache along with other necessary packages:

pip install gptcache onnxruntime openai==0.28 tiktoken

To use the OpenAI API, you need to set up an API key in your environment:

import os
from google.colab import userdata

OPENAI_API_KEY = userdata.get('OPENAI_API_KEY')
os.environ['OPENAI_API_KEY'] = OPENAI_API_KEY

Explanation

  • gptcache is the main library used for caching AI responses.
  • onnxruntime enables fast execution of machine learning models.
  • openai is the official library to interact with OpenAI’s API.
  • The OpenAI API key is retrieved from Google Colab’s userdata module and set as an environment variable.

Real-World Use Case: If your application repeatedly receives the same or similar queries, caching responses prevents unnecessary API calls, reducing costs and improving user experience.

OpenAI API Without GPTCache

Let’s first observe the standard OpenAI API call without caching:

import os
import time
import openai

question = "what’s chatgpt"
openai.api_key = OPENAI_API_KEY
start_time = time.time()
response = openai.ChatCompletion.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": question}],
)

def response_text(openai_resp):
    return openai_resp['choices'][0]['message']['content']

print(f'Question: {question}')
print("Time consuming: {:.2f}s".format(time.time() - start_time))
print(f'Answer: {response_text(response)}\n')

Expected Output

Question: what’s chatgpt
Time consuming: 0.87s
Answer: ChatGPT is a chatbot developed by OpenAI...

Analysis: Every time the same question is asked, an API call is made, leading to additional cost and increased latency.

Implementing GPTCache

To speed up responses, let’s initialize GPTCache:

from gptcache import cache
from gptcache.adapter import openai

cache.init()
cache.set_openai_key()
print("Cache loading...")

Explanation

  • cache.init() initializes the caching system.
  • cache.set_openai_key() sets up the OpenAI API key for GPTCache.

Benefit: Once caching is enabled, repeated queries will return instantly without making API requests.

Query Timing with GPTCache

question = "what's github"
for _ in range(2):
    start_time = time.time()
    response = openai.ChatCompletion.create(
        model='gpt-4o',
        messages=[{"role": "user", "content": question}],
    )
    print(f'Question: {question}')
    print("Time consuming: {:.2f}s".format(time.time() - start_time))
    print(f'Answer: {response_text(response)}\n')

Expected Output

Question: what's github
Time consuming: 0.84s
Answer: GitHub is a web-based platform...

Question: what's github
Time consuming: 0.76s
Answer: GitHub is a web-based platform...

Observation: The second call is significantly faster because GPTCache retrieves the answer without querying the API.

LLM AGENTSRAG PIPELINESTOOL CALLINGDEPLOYMENT
Let's build

Start building AI agents with Build Fast

Explore ProgramJoin Waitlist

Implementing Semantic Search in GPTCache

To enhance caching capabilities, we use similarity-based search with ONNX and FAISS:

from gptcache.embedding import Onnx
from gptcache.manager import CacheBase, VectorBase, get_data_manager
from gptcache.similarity_evaluation.distance import SearchDistanceEvaluation

onnx = Onnx()
data_manager = get_data_manager(CacheBase("sqlite"), VectorBase("faiss", dimension=onnx.dimension))

cache.init(
    embedding_func=onnx.to_embeddings,
    data_manager=data_manager,
    similarity_evaluation=SearchDistanceEvaluation(),
)
cache.set_openai_key()

Explanation

  • ONNX optimizes embedding computation.
  • FAISS accelerates vector search, making similarity-based caching highly efficient.
  • get_data_manager integrates a database (sqlite) and a vector search engine (faiss).

Use Case: If users ask slightly different variations of the same question (e.g., "What is GitHub?", "Tell me about GitHub"), GPTCache retrieves a previously stored response instead of generating a new one.

Exact Match Caching

For applications that require strict matching, GPTCache supports exact match evaluation:

from gptcache.similarity_evaluation.exact_match import ExactMatchEvaluation

cache.init(similarity_evaluation=ExactMatchEvaluation())
cache.set_openai_key()

response = openai.ChatCompletion.create(
    model='gpt-3.5-turbo',
    messages=[{'role': 'user', 'content': 'what is chatgpt'}]
)
print(response)

Benefit

  • Ensures that responses are only retrieved from cache if the query exactly matches a previous query.

Conclusion

GPTCache is a game-changer for optimizing LLM applications, offering significant reductions in API costs and response times. By leveraging exact match caching, semantic search with ONNX and FAISS, and adaptive caching policies, developers can enhance the efficiency of AI applications in production.

Next Steps

  • Experiment with different caching strategies based on your use case.
  • Integrate GPTCache into chatbot applications for improved performance.
  • Explore hybrid caching techniques combining exact match and similarity search.

Resources

  • GPTCache GitHub Repository
  • OpenAI API Documentation
  • ONNX Runtime
  • FAISS
  • GPT Cache Notebook

---------------------------

Stay Updated:- Follow Build Fast with AI pages for all the latest AI updates and resources.

Experts predict 2025 will be the defining year for Gen AI Implementation. Want to be ahead of the curve?

Join Build Fast with AI’s Gen AI Launch Pad 2025 - your accelerated path to mastering AI tools and building revolutionary applications.

---------------------------

Cohort program

Claude MasteryCowork & Code

Explore programJoin waitlistNo coding needed

Resources and Community

Join our community of 12,000+ AI enthusiasts and learn to build powerful AI applications! Whether you're a beginner or an experienced developer, this tutorial will help you understand and implement AI agents in your projects.

  • Website: www.buildfastwithai.com
  • LinkedIn: linkedin.com/company/build-fast-with-ai/
  • Instagram: instagram.com/buildfastwithai/
  • Twitter: x.com/satvikps
  • Telegram: t.me/BuildFastWithAI
Satvik Paramkusham
Written by

Satvik Paramkusham

Founder at Build Fast with AI. AI Engineer, educator, and researcher passionate about agentic AI workflows, LLM application architecture, and training the next generation of AI builders.

Share:

Partner with Build Fast with AI

Guest posts, sponsored articles, tool reviews and in-blog CTAs. See what we offer and get in touch.

Partner with us

Topics

  • Claude AI
  • Best AI Models
  • AI Coding Tools
  • AI Agent Frameworks
  • Agentic AI
  • Open-Source LLMs
  • RAG & Vector Databases
  • AI Industry News

Popular guides

  • 100 Best AI Photo Editing Prompts
  • 75 Best ChatGPT Images 2.5 Prompts
  • 100 Best AI Video Prompts
  • Best AI Models 2026, Tested
  • Claude AI Complete Guide
  • What Is MCP?

Build Fast with AI

  • Build Fast with AI
  • AI Workshops
  • Corporate Training
  • FDE Program
  • Partner with Us
  • Contact
© 2026 Intellify Edventures Private Limited. All rights reserved.
    You Might Also Like
    50 Best Diwali AI Photo Prompts 2026 for Gemini, ChatGPT & Nano Banana
    Tools
    50 Best Diwali AI Photo Prompts 2026 for Gemini, ChatGPT & Nano Banana

    Create stunning Diwali photos with 50 copy-paste AI prompts for Gemini, ChatGPT and Nano Banana. Try realistic portraits, couple photos, saree looks, diyas, rangoli and festive posters.

    Motion Review 2026: The General Video Agent for Editing, Design & Creation
    Tools
    Motion Review 2026: The General Video Agent for Editing, Design & Creation

    Motion review: explore the General Video Agent's AI video editing, motion design, podcast clipping, reference matching, editable timelines, integrations, limitations and use cases.