buildfastwithaibuildfastwithai
AI WorkshopsAll blogsPartner with usFDE Program
Unrot mascotGet yourselfupskilled in AIfor free.Get the appNext live workshopFree AI WorkshopLive session, recording includedReserve a seat

Newsletter

Stay ahead

AI tools and tips. No spam.

Share
Back to blogs
Tools
Tutorials

Guidance: Structured LLM Generation

By Satvik Paramkusham
January 10, 2025
6 min read
Share:
Guidance: Structured LLM Generation
Share:

The best time to start with AI was yesterday. The second best time? Right after reading this post.

Join Build Fast with AI’s Gen AI Launch Pad 2025—a 6-week transformation program designed to accelerate your AI mastery and empower you to build revolutionary applications.

Introduction

Large Language Models (LLMs), such as OpenAI's GPT series, have significantly advanced natural language processing, enabling sophisticated applications across industries. Yet, conventional methods like fine-tuning and prompt engineering can pose challenges, including increased costs, latency, and a lack of precise control over output structure.

Guidance emerges as a game-changer, offering a programming paradigm designed to enhance interaction with LLMs. With features like structured generation, output constraints, and dynamic control logic, Guidance allows developers to optimize performance while reducing operational complexities.

This blog delves into the capabilities of Guidance, providing a step-by-step walkthrough of its features, practical applications, and advanced techniques. By the end, you'll have the tools and knowledge to:

  • Efficiently interact with LLMs.
  • Control and constrain model outputs.
  • Implement multistep workflows for complex tasks.
  • Apply Guidance in real-world scenarios with confidence.

Detailed Explanation

Setting Up Guidance

To begin, ensure your environment is configured correctly. Install the necessary libraries and set up your API key.

!pip install guidance gpustat llama-cpp-python

import os
from google.colab import userdata
os.environ['OPENAI_API_KEY'] = userdata.get('OPENAI_API_KEY')

from guidance import models

gpt4 = models.OpenAI("gpt-4o")

What this does:

  • guidance: The main library enabling structured LLM interactions.
  • gpustat: Tracks GPU usage, useful for monitoring resource consumption in high-performance tasks.
  • llama-cpp-python: Provides integration with LLaMA models for additional flexibility.
  • OPENAI_API_KEY: Authenticates your session with OpenAI's API.

Why it matters: Setting up this environment ensures seamless integration with GPT models, laying the foundation for advanced use cases.

Where to apply: This setup is essential for projects requiring structured LLM interactions, such as chatbots, decision-making systems, or custom workflows.

Simple Generation

A straightforward way to generate responses is by appending a query to a model instance.

lm = gpt4 + "Who won the last Kentucky Derby and by how much?"

Explanation:

  • The + operator combines the model instance (gpt4) with the query string.
  • The model processes the query and generates a response.

Expected Output:

  • A specific answer detailing the winner of the Kentucky Derby and the margin of victory.

Use Case: Ideal for single-turn Q&A systems, basic chatbot applications, or quick information retrieval.

Pro Tip: Experiment with varying query structures to understand how phrasing impacts the model’s responses.

Making Decisions with select

Guidance introduces the ability to choose between predefined alternatives, enabling dynamic responses based on user input.

from guidance import select

gpt4 + f'''
Q: {query}
Now I will choose to either SEARCH the web or RESPOND.
Choice: {select(["SEARCH", "RESPOND"], name="choice")}
'''

Explanation:

  • select: Allows the model to choose between predefined options.
  • The variable choice stores the model’s selection, influencing subsequent actions.

Expected Output:

  • If the model chooses “SEARCH”: It indicates the need for external input.
  • If it chooses “RESPOND”: It directly generates an answer.

Applications: This is particularly useful for scenarios like:

  • Decision trees in interactive chatbots.
  • Routing logic in customer support systems.
  • Dynamic branching workflows.

Interleaved Generation and Control

Guidance excels in blending logic with text generation, enabling sophisticated control over outputs. Here’s an example:

@guidance
def qa_bot(lm, query):
    lm += f'''
    Q: {query}
    Now I will choose to either SEARCH the web or RESPOND.
    Choice: {select(["SEARCH", "RESPOND"], name="choice")}
    '''
    if lm["choice"] == "SEARCH":
        lm += "A: I don't know, Google it!"
    else:
        lm += f'A: {gen(stop="Q:", name="answer")}'
    return lm

gpt4 + qa_bot(query)

Explanation:

  • guidance decorator: Marks the function as a Guidance-enabled workflow.
  • gen: Dynamically generates text with defined constraints (e.g., stop="Q:").
  • Conditional logic determines whether to defer to external resources or generate a direct response.

Expected Output:

  • For “SEARCH”: A suggestion to consult external sources.
  • For “RESPOND”: A complete answer.

Applications: Build chatbots or assistants that adaptively handle queries based on their complexity or the availability of information.

Regex-Guided Output

Regex constraints ensure outputs adhere to specific formats, which is invaluable for structured data generation.

gpt4o + f'''
Tweak this proverb to apply to model instructions instead.

Where there is no guidance, a people falls,
but in an abundance of counselors there is safety.
- Proverbs 11:14

UPDATED
Where there is no guidance{gen('rewrite', stop="- ")}
- GPT {gen('chapter', regex="[0-9]+")}:{gen('verse', regex="[0-9]+")}
'''

Explanation:

  • regex: Ensures generated outputs match specific patterns (e.g., numerical values for chapter and verse).
  • The gen function dynamically generates text adhering to these constraints.

Expected Output: A rephrased proverb, properly formatted with new chapter and verse references.

Applications:

  • Academic citations.
  • Template-based text generation.
  • Ensuring data consistency in structured datasets.

Pro Tip: Experiment with more complex regex patterns to enforce advanced constraints.

Multistep Interaction

Here’s how Guidance can handle multistep processes with conditional branching:

@guidance
def experts(lm, query):
    with system():
        lm += "You are a helpful assistant."

    with user():
        lm += f'''
        I want a response to the following question:
        {query}
        Who are 3 world-class experts (past or present) who would be great at answering this?
        Please don't answer the question or comment on it yet.'''

    with assistant():
        lm += gen(name='experts', max_tokens=300)

    with user():
        lm += f'''
        Great, now please answer the question as if these experts had collaborated in writing a joint anonymous answer.
        If the experts would disagree, just present their different positions as alternatives.'''

    with assistant():
        lm += gen(name='answer', max_tokens=500)

    return lm

gpt4 + experts(query='What is the meaning of life?')

Explanation:

  • Context Blocks:
  • system: Provides initial setup or context.
  • user: Captures user queries or instructions.
  • assistant: Generates responses based on the above inputs.
  • gen: Dynamically generates expert recommendations and a collaborative response.

Expected Output:

  • A list of three world-class experts.
  • A nuanced answer reflecting diverse perspectives.

Applications: Use this approach for research, consulting, or expert-driven content creation.

Conclusion

Guidance transforms the way developers interact with LLMs, offering unparalleled control and flexibility. By mastering its tools—from regex constraints to dynamic workflows—you can unlock new possibilities in text generation while optimizing for efficiency and cost.

Resources

  • Guidance Documentation
  • OpenAI GPT API
  • Regex Cheat Sheet
  • Guidance Gpustat Build Fast With AI NoteBook

---------------------------------

Stay Updated:- Follow Build Fast with AI pages for all the latest AI updates and resources.

Experts predict 2025 will be the defining year for Gen AI implementation.Want to be ahead of the curve?

Join Build Fast with AI’s Gen AI Launch Pad 2025 - your accelerated path to mastering AI tools and building revolutionary applications.


Satvik Paramkusham
Written by

Satvik Paramkusham

Founder at Build Fast with AI. AI Engineer, educator, and researcher passionate about agentic AI workflows, LLM application architecture, and training the next generation of AI builders.

Share:

Partner with Build Fast with AI

Guest posts, sponsored articles, tool reviews and in-blog CTAs. See what we offer and get in touch.

Partner with us

Topics

  • Claude AI
  • Best AI Models
  • AI Coding Tools
  • AI Agent Frameworks
  • Agentic AI
  • Open-Source LLMs
  • RAG & Vector Databases
  • AI Industry News

Popular guides

  • 100 Best AI Photo Editing Prompts
  • 75 Best ChatGPT Images 2.5 Prompts
  • 100 Best AI Video Prompts
  • Best AI Models 2026, Tested
  • Claude AI Complete Guide
  • What Is MCP?

Build Fast with AI

  • Build Fast with AI
  • AI Workshops
  • Corporate Training
  • FDE Program
  • Partner with Us
  • Contact
© 2026 Intellify Edventures Private Limited. All rights reserved.
    You Might Also Like
    ChatGPT Images 2.6: Features, Pricing and Review
    Tutorials
    ChatGPT Images 2.6: Features, Pricing and Review

    ChatGPT Images 2.6 review covering Flare vs Sunburst, GIFs, SVG output, image editing, API pricing and practical prompts for creators.

    50 Best Diwali AI Photo Prompts 2026 for Gemini, ChatGPT & Nano Banana
    Tools
    50 Best Diwali AI Photo Prompts 2026 for Gemini, ChatGPT & Nano Banana

    Create stunning Diwali photos with 50 copy-paste AI prompts for Gemini, ChatGPT and Nano Banana. Try realistic portraits, couple photos, saree looks, diyas, rangoli and festive posters.