DSPy (Data Science Prompting) is a framework designed to streamline the process of working with language models. It shifts the focus from "prompting" (manually crafting queries) to "programming" (building modular AI systems).
It offers tools for:
- Modular design.
- Optimizing prompts and weights.
- Creating systems like classifiers, retrieval-augmented generation (RAG) pipelines, and agent loops.
Setup and Installation
1.Install Required Libraries
pip install httpx==0.27.2 dspy faiss-cpu
- httpx : A high-performance HTTP client for Python.
- dspy : The core library for building AI systems with DSPy.
- faiss-cpu :A library for efficient similarity search and clustering, often used in RAG task
2.Configure OpenAI API
import os
from google.colab import userdata
os.environ['OPENAI_API_KEY'] = userdata.get('OPENAI_API_KEY')
OPENAIKEY = os.getenv('OPENAI_API_KEY')
- Sets up the OpenAI API key. This key is required to access GPT-based models.
- userdata.get() fetches the key securely (useful in Colab environments).
3.Setting Up DSPy
import dspy lm = dspy.OpenAI(model='gpt-4o', api_key=OPENAIKEY, model_type='chat', max_tokens=500) dspy.settings.configure(lm=lm)
- Initializes DSPy with OpenAI's GPT-4o model.
- Configures DSPy to use this model for all subsequent tasks.
Loading the Dataset
- Dataset: HotPotQA, a popular dataset for multi-hop question answering.
- Code:
from dspy.datasets import HotPotQA
# Load the dataset.
dataset = HotPotQA(train_seed=1, train_size=20, eval_seed=2023, dev_size=50, test_size=0)
trainset = [x.with_inputs('question') for x in dataset.train]
devset = [x.with_inputs('question') for x in dataset.dev]
- HotPotQA() : Loads a dataset with a specific configuration.
- train_size=2 0 : Use 20 examples for training.
- dev_size=50 : Use 50 examples for development (validation).
- test_size=0 : No examples for testing in this case.
- x.with_inputs('question') : Processes each example to focus on the "question" field.
Inspecting a Training Example
train_example = trainset[0]
print(train_example)
print(f"Question: {train_example.question}")
print(f"Answer: {train_example.answer}")
- trainset[0] : Fetches the first training example.
- Displays the question and the expected answer for inspection.
Defining the BasicQA Signature
class BasicQA(dspy.Signature):
"""Answer questions with short factoid answers."""
question = dspy.InputField()
answer = dspy.OutputField(desc="often between 1 and 5 words")
Signature: A blueprint for specifying inputs and outputs of a module.
Components:
- question : Defines the input field.
- answer : Defines the output field with a description.
Creating the BasicQABot Module
class BasicQABot(dspy.Module):
def __init__(self):
super().__init__()
self.generate = dspy.Predict(BasicQA)
def forward(self, question):
prediction = self.generate(question=question)
return dspy.Prediction(answer=prediction.answer)
- BasicQABot:
- A simple question-answering bot.
__init__:- Initializes the BasicQABot with a prediction model (self.generate).
forward():- Takes a question as input.
- Uses self.generate() to predict an answer.
- Returns the predicted answer.
Querying the QA Bot
qa_bot = BasicQABot()
pred = qa_bot.forward("In the 10th Century A.D. Ealhswith had a son called Æthelweard by which English king?")
print(pred.answer)
- Functionality:
- Creates an instance of BasicQABot.
- Queries it with a historical question.
- Outputs the predicted answer.
Retrieval-Augmented Generation (RAG)
- RAG: Combines retrieval systems (to fetch relevant context) with generative models (to generate responses).
Configuring DSPy for RAG
import dspy
lm = dspy.LM('openai/gpt-4o-mini')
dspy.configure(lm=lm)
- Configures DSPy to use a smaller version of GPT-4o.
Simple QA Chain
qa = dspy.Predict('question: str -> response: str')
response = qa(question="what are high memory and low memory on linux?")
print(response.response)
- QA Chain:
- Defines a simple chain that maps a question string to a response string.
- Predicts the response using the QA chain.
Chain Of Thought Module
cot = dspy.ChainOfThought('question -> response')
cot(question="should curly braces appear on their own line?")
- Chain of Thought (CoT):
- Models multi-step reasoning processes.
- Takes a question and outputs both reasoning and response.
Manipulating Examples in DSPy
data = [dspy.Example(**d).with_inputs('question') for d in data]
dspy.Example:- Converts raw JSON data into DSPy-compatible examples.
- Focuses on the question field for processing.
Evaluation in DSPy
Semantic F1 Metric
from dspy.evaluate import SemanticF1 metric = SemanticF1(decompositional=True) score = metric(example, pred)
- Semantic F1:
- Measures similarity between the predicted and gold-standard responses.
- Useful for evaluating generated answers.
Using DSPy for Math Reasoning
- Dataset: MATH (Mathematical reasoning benchmark).
- Workflow: - Load the dataset
- Use a CoT module for reasoning.
- Evaluate predictions using DSPy utilities.
Conclusion:- DSPy is a powerful framework for building modular AI systems, streamlining the process of programming with language models. From basic question-answering bots to advanced Retrieval-Augmented Generation (RAG) pipelines, DSPy offers tools and algorithms to optimize prompts, weights, and workflows efficiently. Its flexibility allows developers to iterate rapidly, evaluate models effectively, and enhance performance with integrated metrics like Semantic F1. Whether you're working on natural language processing, reasoning tasks, or AI-driven applications, DSPy simplifies complex implementations, empowering developers to unlock the full potential of AI.
Resources
--------------
Stay Updated:- Follow Build Fast with AI pages for all the latest AI updates and resources.
Experts predict 2025 will be the defining year for Gen AI implementation.Want to be ahead of the curve?
Join Build Fast with AI’s Gen AI Launch Pad 2025 - your accelerated path to mastering AI tools and building revolutionary applications.
👉 Limited Spots, join the waitlist now: www.buildfastwithai.com/genai-course
