Calls | Mirascope
MirascopeLilypad

Calls

If you haven't already, we recommend first reading the section on writing Prompts

When working with Large Language Model (LLM) APIs in Mirascope, a "call" refers to making a request to a LLM provider's API with a particular setting and prompt.

The call decorator is a core feature of the Mirascope library, designed to simplify and streamline interactions with various LLM providers. This powerful tool allows you to transform prompt templates written as Python functions into LLM API calls with minimal boilerplate code while providing type safety and consistency across different providers.

We currently support OpenAI, Anthropic, Google (Gemini/Vertex), Groq, xAI, Mistral, Cohere, LiteLLM, Azure AI, and Amazon Bedrock.

If there are any providers we don't yet support that you'd like to see supported, let us know!

Basic Usage and Syntax

Let's take a look at a basic example using Mirascope vs. official provider SDKs:

Mirascope Frog Logo

Mirascope

from mirascope import llm


@llm.call(provider="openai", model="gpt-4o-mini") 
def recommend_book(genre: str) -> str:
    return f"Recommend a {genre} book"


response: llm.CallResponse = recommend_book("fantasy")
print(response.content)

Official provider SDKs typically require more boilerplate code:

Official SDK

Notice how Mirascope makes calls more readable by reducing boilerplate and standardizing interactions with LLM providers.

The llm.call decorator accepts provider and model arguments and returns a provider-agnostic CallResponse instance that provides a consistent interface regardless of the underlying provider. You can find more information on CallResponse in the section below on handling responses.

Note the @prompt_template decorator is optional unless you're using string templates.

Runtime Provider Overrides

You can override provider settings at runtime using llm.override. This takes a function decorated with llm.call and lets you specify:

  • provider: Change the provider being called
  • model: Use a different model
  • call_params: Override call parameters like temperature
  • client: Use a different client instance

When overriding with a specific provider, you must specify the model parameter.

from mirascope import llm


@llm.call(provider="openai", model="gpt-4o-mini") 
def recommend_book(genre: str) -> str: 
    return f"Recommend a {genre} book"


response: llm.CallResponse = recommend_book("fantasy")
print(response.content)

override_response = llm.override( 
    recommend_book, 
    provider="anthropic", 
    model="claude-3-5-sonnet-latest", 
    call_params={"temperature": 0.7}, 
)("fantasy") 

print(override_response.content)

Handling Responses

Common Response Properties and Methods

All BaseCallResponse objects share these common properties:

  • content: The main text content of the response. If no content is present, this will be the empty string.
  • finish_reasons: A list of reasons why the generation finished (e.g., "stop", "length"). These will be typed specifically for the provider used. If no finish reasons are present, this will be None.
  • model: The name of the model used for generation.
  • id: A unique identifier for the response if available. Otherwise this will be None.
  • usage: Information about token usage for the call if available. Otherwise this will be None.
  • input_tokens: The number of input tokens used if available. Otherwise this will be None.
  • output_tokens: The number of output tokens generated if available. Otherwise this will be None.
  • cost: An estimated cost of the API call if available. Otherwise this will be None.
  • message_param: The assistant's response formatted as a message parameter.
  • tools: A list of provider-specific tools used in the response, if any. Otherwise this will be None. Check out the Tools documentation for more details.
  • tool: The first tool used in the response, if any. Otherwise this will be None. Check out the Tools documentation for more details.
  • tool_types: A list of tool types used in the call, if any. Otherwise this will be None.
  • prompt_template: The prompt template used for the call.
  • fn_args: The arguments passed to the function.
  • dynamic_config: The dynamic configuration used for the call.
  • metadata: Any metadata provided using the dynamic configuration.
  • messages: The list of messages sent in the request.
  • call_params: The call parameters provided to the call decorator.
  • call_kwargs: The finalized keyword arguments used to make the API call.
  • user_message_param: The most recent user message, if any. Otherwise this will be None.
  • start_time: The timestamp when the call started.
  • end_time: The timestamp when the call ended.

There are also two common methods:

  • __str__: Returns the content property of the response for easy printing.
  • tool_message_params: Creates message parameters for tool call results. Check out the Tools documentation for more information.

Multi-Modal Outputs

While most LLM providers focus on text outputs, some providers support additional output modalities like audio. The availability of multi-modal outputs varies among providers:

ProviderTextAudioImage
OpenAI✓✓-
Anthropic✓--
Mistral✓--
Google Gemini✓--
Groq✓--
Cohere✓--
LiteLLM✓--
Azure AI✓--
Legend: ✓ (Supported), - (Not Supported)

Audio Outputs

  • audio: Configuration for the audio output (voice, format, etc.)
  • modalities: List of output modalities to receive (e.g. ["text", "audio"])

For providers that support audio outputs, you can receive both text and audio responses from your calls:

When using models that support audio outputs, you'll have access to:

  • content: The text content of the response
  • audio: The raw audio bytes of the response
  • audio_transcript: The transcript of the audio response

Common Parameters Across Providers

There are several common parameters that you'll find across all providers when using the call decorator. These parameters allow you to control various aspects of the LLM call:

  • model: The only required parameter for all providers, which may be passed in as a standard argument (whereas all others are optional and must be provided as keyword arguments). It specifies which language model to use for the generation. Each provider has its own set of available models.
  • stream: A boolean that determines whether the response should be streamed or returned as a complete response. We cover this in more detail in the Streams documentation.
  • response_model: A Pydantic BaseModel type that defines how to structure the response. We cover this in more detail in the Response Models documentation.
  • output_parser: A function for parsing the response output. We cover this in more detail in the Output Parsers documentation.
  • json_mode: A boolean that deterines whether to use JSON mode or not. We cover this in more detail in the JSON Mode documentation.
  • tools: A list of tools that the model may request to use in its response. We cover this in more detail in the Tools documentation.
  • client: A custom client to use when making the call to the LLM. We cover this in more detail in the Custom Client section below.
  • call_params: The provider-specific parameters to use when making the call to that provider's API. We cover this in more detail in the Provider-Specific Usage section below.

These common parameters provide a consistent way to control the behavior of LLM calls across different providers. Keep in mind that while these parameters are widely supported, there might be slight variations in how they're implemented or their exact effects across different providers (and the documentation should cover any such differences).

Since call_params is just a TypedDict, you can always include any additional keys at the expense of type errors (and potentially unknown behavior). This presents one way to pass provider-specific parameters (or deprecated parameters) while still using the general interface.

from mirascope import llm


@llm.call(provider="openai", model="gpt-4o-mini", call_params={"max_tokens": 512}) 
def recommend_book(genre: str) -> str:
    return f"Recommend a {genre} book"


response: llm.CallResponse = recommend_book("fantasy")
print(response.content)

Dynamic Configuration

Often you will want (or need) to configure your calls dynamically at runtime. Mirascope supports returning a BaseDynamicConfig from your prompt template, which will then be used to dynamically update the settings of the call.

In all cases, you will need to return your prompt messages through the messages keyword of the dynamic config unless you're using string templates.

Call Params

from mirascope import BaseDynamicConfig, Messages, llm


@llm.call(provider="openai", model="gpt-4o-mini")
def recommend_book(genre: str) -> BaseDynamicConfig:
    return {
        "messages": [Messages.User(f"Recommend a {genre} book")], 
        "call_params": {"max_tokens": 512}, 
        "metadata": {"tags": {"version:0001"}},
    }


response: llm.CallResponse = recommend_book("fantasy")
print(response.content)

Metadata

from mirascope import BaseDynamicConfig, Messages, llm


@llm.call(provider="openai", model="gpt-4o-mini")
def recommend_book(genre: str) -> BaseDynamicConfig:
    return {
        "messages": [Messages.User(f"Recommend a {genre} book")], 
        "call_params": {"max_tokens": 512},
        "metadata": {"tags": {"version:0001"}}, 
    }


response: llm.CallResponse = recommend_book("fantasy")
print(response.content)

Provider-Specific Usage

While Mirascope provides a consistent interface across different LLM providers, you can also use provider-specific modules with refined typing for an individual provider.

When using the provider modules, you'll receive a provider-specific BaseCallResponse object, which may have extra properties. Regardless, you can always access the full, provider-specific response object as response.response.

Custom Messages

When using provider-specific calls, you can also always return the original message types for that provider. To do so, simply return the provider-specific dynamic config:

Support for provider-specific messages ensures that you can still access newly released provider-specific features that Mirascope may not yet support natively.

Custom Client

Mirascope allows you to use custom clients when making calls to LLM providers. This feature is particularly useful when you need to use specific client configurations, handle authentication in a custom way, or work with self-hosted models.

Decorator Parameter:

You can pass a client to the call decorator using the client parameter:

Dynamic Configuration:

You can also configure the client dynamically at runtime through the dynamic configuration:

Error Handling

When making LLM calls, it's important to handle potential errors. Mirascope preserves the original error messages from providers, allowing you to catch and handle them appropriately:

from mirascope import llm


@llm.call(provider="openai", model="gpt-4o-mini")
def recommend_book(genre: str) -> str:
    return f"Recommend a {genre} book"


try:
    response: llm.CallResponse = recommend_book("fantasy") 
    print(response.content)
except Exception as e:
    print(f"Error: {str(e)}")

These examples catch the base Exception class; however, you can (and should) catch provider-specific exceptions instead when using provider-specific modules.

Next Steps

By mastering calls in Mirascope, you'll be well-equipped to build robust, flexible, and reusable LLM applications.

Next, we recommend choosing one of:

  • Streams to see how to stream call responses for a more real-time interaction.
  • Chaining to see how to chain calls together.
  • Response Models to see how to generate structured outputs.
  • Tools to see how to give LLMs access to custom tools to extend their capabilities.
  • Async to see how to better take advantage of asynchronous programming and parallelization for improved performance.

Pick whichever path aligns best with what you're hoping to get from Mirascope.