What’s the limit of AI’s potential?
At Gen AI Launch Pad 2024, redefine what’s possible. Step up and be the pioneer shaping the limitless future of AI.
Introduction
In today's rapidly evolving AI landscape, managing multiple AI models and providers can be challenging for developers. Portkey AI emerges as a powerful solution, offering a unified interface for integrating and managing over 250 AI models. This comprehensive guide will walk you through setting up Portkey AI and leveraging its powerful features for your applications.
What is Portkey AI?
Portkey AI serves as a comprehensive platform that streamlines AI integration for developers and organizations. Think of it as a universal remote control for AI services - one interface to manage them all. The platform offers:
- Access to 250+ AI models
- Advanced monitoring and logging capabilities
- Structured output handling
- Multi-provider support
- Cost and usage tracking
Getting Started with Portkey
Installation and Setup
First, let's install the necessary packages:
!pip install -qU portkey-ai openai portkey
Next, set up your API keys:
import os
from google.colab import userdata
os.environ['OPENAI_API_KEY'] = userdata.get('OPENAI_API_KEY')
os.environ['PORTKEY_API_KEY'] = userdata.get('PORTKEY_API_KEY')
Basic Integration
Here's how to set up the Portkey gateway with OpenAI:
from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders
client = OpenAI(
api_key=userdata.get('OPENAI_API_KEY'),
base_url=PORTKEY_GATEWAY_URL,
default_headers=createHeaders(
provider="openai",
api_key=userdata.get('PORTKEY_API_KEY')
)
)
Advanced Features
1. User Tracking
Portkey allows you to monitor individual user interactions and costs:
chat_complete = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "What is Portkey-AI?"}],
user="shubham"
)
2. Structured Outputs
Portkey supports enforcing specific JSON schemas for model outputs:
from pydantic import BaseModel
class Step(BaseModel):
explanation: str
output: str
class MathReasoning(BaseModel):
steps: list[Step]
final_answer: str
completion = client.beta.chat.completions.parse(
model="gpt-4o",
messages=[
{"role": "system", "content": "You are a helpful math tutor."},
{"role": "user", "content": "how can I solve 8x + 7 = -23"}
],
response_format=MathReasoning,
)
3. Multi-Provider Support
Anthropic Integration
client = OpenAI(
api_key=userdata.get('ANTHROPIC_API_KEY'),
base_url=PORTKEY_GATEWAY_URL,
default_headers=createHeaders(
provider="anthropic",
api_key=userdata.get("PORTKEY_API_KEY")
),
)
Mistral AI Integration
client = OpenAI(
api_key=userdata.get('MISTRAL_API_KEY'),
base_url=PORTKEY_GATEWAY_URL,
default_headers=createHeaders(
provider="mistral-ai",
api_key=userdata.get("PORTKEY_API_KEY")
)
)
Best Practices and Tips
Logging and Monitoring
- Use the Portkey dashboard to monitor:
- Token usage
- Execution time
- Cost per request
- User-level analytics
Error Handling
Always implement proper error handling:
try:
response = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "Hello"}]
)
except Exception as e:
print(f"Error: {e}")
Practical Applications
1. Customer Support Automation
Use Portkey to build a support system that can:
- Route queries to appropriate AI models
- Track user interactions
- Maintain conversation context
- Generate structured responses
2. Content Generation Pipeline
Create a content generation system that:
- Uses different models for different content types
- Ensures output consistency through schemas
- Tracks usage and costs per content piece
3. Data Analysis Workflow
Implement an analysis pipeline that:
- Processes data through multiple AI models
- Generates structured reports
- Maintains audit trails of all operations
Future Considerations
As AI technology evolves, Portkey is positioned to:
- Support new AI models and providers
- Enhance monitoring capabilities
- Improve cost optimization features
- Expand integration options
Conclusion
Portkey AI represents a significant step forward in making AI integration more accessible and manageable for developers. By providing a unified interface, robust monitoring tools, and support for multiple providers, it simplifies the complex landscape of AI service integration. Whether you're building a simple chatbot or a complex AI-powered application, Portkey offers the tools and features needed to succeed.
Remember to regularly check the official documentation and community resources as new features and capabilities are added to the platform. Happy coding!
Resources and Further Reading
Community Resources
---------------------------------
Stay Updated:- Follow Build Fast with AI pages for all the latest AI updates and resources.
Experts predict 2025 will be the defining year for Gen AI implementation.Want to be ahead of the curve?
Join Build Fast with AI’s Gen AI Launch Pad 2025 - your accelerated path to mastering AI tools and building revolutionary applications.
