Prompt Integration

Use prompts stored in Helicone with the AI Gateway

Overview

Helicone AI Gateway integrates seamlessly with Helicone’s prompt management system, allowing you to:

• Use versioned prompts stored in Helicone
• Dynamically inject prompt variables
• Track prompt usage across providers
• Deploy prompts without code changes

Prompt integration works with all gateway features including routing, fallbacks, and BYOK/PTB.

How It Works

When you include prompt fields in your request, the gateway:

1. Fetches the prompt template from Helicone
2. Injects your input variables
3. Expands the template into messages
4. Routes to the appropriate provider

Using Prompts with the Gateway

Basic Prompt Usage

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://ai-gateway.helicone.ai",
  apiKey: process.env.HELICONE_API_KEY,
});

const response = await client.chat.completions.create({
  prompt_id: "pirate-bot",
  model: "gpt-4o-mini",
  inputs: {
    person: "Alice",
  },
});

What happens:

1. Gateway fetches the “pirate-bot” prompt template
2. Injects person: "Alice" into the template
3. Expands to full messages array
4. Routes to GPT-4o-mini

Model from Prompt

You can omit the model field and use the model defined in the prompt:

const response = await client.chat.completions.create({
  prompt_id: "pirate-bot",
  inputs: {
    person: "Alice",
  },
});
// Uses the model configured in the "pirate-bot" prompt

If both model and prompt_id are provided, the model in the request takes precedence.

Prompt Versioning

Use specific prompt versions:

const response = await client.chat.completions.create({
  prompt_id: "pirate-bot",
  version_id: "v2.1.3",  // Use specific version
  model: "gpt-4o-mini",
  inputs: {
    person: "Alice",
  },
});

Environment-Specific Prompts

Use different prompt versions per environment:

const response = await client.chat.completions.create({
  prompt_id: "pirate-bot",
  environment: "production",  // or "staging", "development"
  model: "gpt-4o-mini",
  inputs: {
    person: "Alice",
  },
});

Prompt Fields

prompt_id · string · required

The ID of the prompt to use from Helicone

inputs · object · required

Key-value pairs to inject into the prompt template

version_id · string

Specific version of the prompt to use (default: latest)

environment · string

Environment to use for prompt resolution: production, staging, or development

model · string

Override the model defined in the prompt

Creating Prompts

Create and manage prompts in the Helicone dashboard:

1. Navigate to Prompts

   Go to Prompts in the Helicone dashboard

2. Create New Prompt

   Click “New Prompt” and define:

   • Prompt ID (e.g., “pirate-bot”)
   • Model to use
   • Message template with variables
   • Version tags
3. Add Variables

   Use {{ variable_name }} syntax in your prompt:

   You are a helpful chatbot that only talks like a pirate.
   You are speaking with {{ person }}!
   
4. Deploy

   Save and deploy your prompt

Prompt Format with hpf

Use the Helicone Prompt Format (hpf) helper in your code:

import { hpf } from "@helicone/prompts";

const chatCompletion = await client.chat.completions.create(
  {
    model: "gpt-4-turbo",
    messages: [
      {
        role: "system",
        content: hpf`You are a helpful chatbot, that only talks like a pirate.
        You are speaking with ${{ person: "Alice" }}!`,
      },
    ],
    max_tokens: 700,
  },
  {
    headers: {
      "Helicone-Prompt-Id": "pirate-bot",
    },
  }
);

The hpf helper automatically tracks prompt variables and associates them with the prompt ID.

Prompts with Fallbacks

Combine prompts with provider fallbacks:

const response = await client.chat.completions.create({
  prompt_id: "pirate-bot",
  model: "gpt-4o/openai,gpt-4o/azure,claude-sonnet-4",
  inputs: {
    person: "Alice",
  },
});

Behavior:

1. Fetches “pirate-bot” prompt
2. Tries GPT-4o on OpenAI
3. Falls back to Azure if needed
4. Falls back to Claude if needed

Using with Helicone-Auth Header

When using the traditional proxy pattern with Helicone-Auth header:

const client = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,  // Provider key
  baseURL: "https://oai.helicone.ai/v1",
  defaultHeaders: {
    "Helicone-Auth": `Bearer ${process.env.HELICONE_API_KEY}`,
  },
});

const response = await client.chat.completions.create(
  {
    model: "gpt-4-turbo",
    messages: [
      {
        role: "system",
        content: hpf`You are a helpful chatbot.
        Speaking with ${{ person: "Alice" }}!`,
      },
    ],
  },
  {
    headers: {
      "Helicone-Prompt-Id": "pirate-bot",
    },
  }
);

Prompt Tracking

All requests using prompts are automatically tracked:

1. View Prompt Usage

   Navigate to Prompts in the dashboard

2. Select Your Prompt

   Click on a prompt to see:

   • Total requests
   • Success rate
   • Cost by provider
   • Latency metrics
3. Analyze Performance

   Compare versions and environments:

   • Which version performs best?
   • Which provider is most cost-effective?
   • Where are errors occurring?

Advanced Patterns

Dynamic Model Selection

Use prompt fields with dynamic models:

function getPromptConfig(userTier: "free" | "premium") {
  return {
    prompt_id: "pirate-bot",
    model: userTier === "premium" 
      ? "gpt-4o" 
      : "gpt-4o-mini/deepinfra,gpt-4o-mini",
  };
}

const response = await client.chat.completions.create({
  ...getPromptConfig("free"),
  inputs: {
    person: "Alice",
  },
});

Conditional Prompt Selection

Select prompts based on context:

function selectPrompt(conversation_type: string) {
  const prompts = {
    customer_support: "support-bot",
    sales: "sales-assistant",
    technical: "tech-expert",
  };
  return prompts[conversation_type] || "default-bot";
}

const response = await client.chat.completions.create({
  prompt_id: selectPrompt("customer_support"),
  model: "gpt-4o-mini",
  inputs: {
    user_name: "Alice",
    issue: "billing question",
  },
});

A/B Testing Prompts

Test different prompt versions:

function getPromptVersion(userId: string) {
  // Simple A/B test: 50/50 split
  const hash = userId.charCodeAt(0);
  return hash % 2 === 0 ? "v1.0" : "v2.0";
}

const response = await client.chat.completions.create({
  prompt_id: "pirate-bot",
  version_id: getPromptVersion(user.id),
  model: "gpt-4o-mini",
  inputs: {
    person: user.name,
  },
});

Error Handling

Prompt Not Found

{
  "error": {
    "message": "Prompt 'invalid-id' not found",
    "type": "prompt_not_found"
  }
}

Missing Required Inputs

{
  "error": {
    "message": "Missing required input: 'person'",
    "type": "missing_input"
  }
}

Invalid Version

{
  "error": {
    "message": "Version 'v999' not found for prompt 'pirate-bot'",
    "type": "version_not_found"
  }
}

Best Practices

Version Your Prompts

Use semantic versioning for prompts:

• v1.0.0: Major changes (breaking)
• v1.1.0: Minor improvements
• v1.1.1: Bug fixes

Pin critical production code to specific versions:

version_id: "v1.2.3"  // Stable version

Use Environment Tags

Deploy different prompts per environment:

environment: process.env.NODE_ENV  // production, staging, development

Track Prompt Performance

Monitor prompt metrics in the dashboard:

• Success rate by version
• Cost per prompt
• Latency trends
• Provider distribution

Test Before Deploying

Test prompt changes thoroughly:

1. Deploy to development environment
2. Test with various inputs
3. Promote to staging
4. Deploy to production after validation

Use Descriptive IDs

Use clear, descriptive prompt IDs:

// Good
prompt_id: "customer-support-greeting"
prompt_id: "sales-product-recommendation"

// Avoid
prompt_id: "prompt1"
prompt_id: "test"

Complete Example

import OpenAI from "openai";
import { hpf } from "@helicone/prompts";

const client = new OpenAI({
  baseURL: "https://ai-gateway.helicone.ai",
  apiKey: process.env.HELICONE_API_KEY,
});

async function customerSupportBot(userName: string, issue: string) {
  try {
    const response = await client.chat.completions.create({
      prompt_id: "customer-support",
      environment: process.env.NODE_ENV,
      model: "gpt-4o/openai,gpt-4o/azure,claude-sonnet-4",
      inputs: {
        user_name: userName,
        issue: issue,
        current_date: new Date().toISOString(),
      },
    });
    
    return response.choices[0].message.content;
  } catch (error) {
    console.error("Support bot error:", error);
    throw error;
  }
}

// Usage
const reply = await customerSupportBot(
  "Alice",
  "I can't access my account"
);
console.log(reply);

Next Steps

Create Prompts

Start creating prompts in the dashboard

Routing

Learn about provider routing

Fallbacks

Configure automatic failover

Prompt Docs

Full prompt management documentation