llm

Class APIError

Base class for API-related errors.

Bases:

Attribute AssistantContent

Type: TypeAlias

Type alias for content that can fit into an AssistantMessage.

Attribute AssistantContentChunk

Type: TypeAlias

Chunks of assistant content that may be streamed as generated by the LLM.

Attribute AssistantContentPart

Type: TypeAlias

Content parts that can be included in an AssistantMessage.

Class AssistantMessage

An assistant message containing the model's response.

Attribute AsyncChunkIterator

Type: TypeAlias

Asynchronous iterator yielding chunks with raw data.

Class AsyncContextResponse

The response generated by an LLM from an async context call.

Bases: BaseResponse[AsyncContextToolkit[DepsT], FormattableT], Generic[DepsT, FormattableT]

Function execute_tools

Execute and return all of the tool calls in the response.

Function resume

Generate a new AsyncContextResponse using this response's messages with additional user content.

Uses this response's tools and format type. Also uses this response's provider, model, client, and params, unless the model context manager is being used to provide a new LLM as an override.

Class AsyncContextStreamResponse

An AsyncContextStreamResponse wraps response content from the LLM with a streaming interface.

This class supports iteration to process chunks as they arrive from the model.

Content can be streamed in one of three ways:

• Via .streams(), which provides an iterator of streams, where each stream contains chunks of streamed data. The chunks contain deltas (new content in that particular chunk), and the stream itself accumulates the collected state of all the chunks processed thus far.
• Via .chunk_stream() which allows iterating over Mirascope's provider- agnostic chunk representation.
• Via .pretty_stream() a helper method which provides all response content as str deltas. Iterating through pretty_stream will yield text content and optionally placeholder representations for other content types, but it will still consume the full stream.
• Via .structured_stream(), a helper method which provides partial structured outputs from a response (useful when FormatT is set). Iterating through structured_stream will only yield structured partials, but it will still consume the full stream.

As chunks are consumed, they are collected in-memory on the AsyncContextStreamResponse, and they become available in .content, .messages, .tool_calls, etc. All of the stream iterators can be restarted after the stream has been consumed, in which case they will yield chunks from memory in the original sequence that came from the LLM. If the stream is only partially consumed, a fresh iterator will first iterate through in-memory content, and then will continue consuming fresh chunks from the LLM.

In the specific case of text chunks, they are included in the response content as soon as they become available, via an llm.Text part that updates as more deltas come in. This enables the behavior where resuming a partially-streamed response will include as much text as the model generated.

For other chunks, like Thinking or ToolCall, they are only added to response content once the corresponding part has fully streamed. This avoids issues like adding incomplete tool calls, or thinking blocks missing signatures, to the response.

For each iterator, fully iterating through the iterator will consume the whole LLM stream. You can pause stream execution midway by breaking out of the iterator, and you can safely resume execution from the same iterator if desired.

Bases: BaseAsyncStreamResponse[AsyncContextToolkit, FormattableT], Generic[DepsT, FormattableT]

Function execute_tools

Execute and return all of the tool calls in the response.

Function resume

Generate a new AsyncContextStreamResponse using this response's messages with additional user content.

Uses this response's tools and format type. Also uses this response's provider, model, client, and params, unless the model context manager is being used to provide a new LLM as an override.

Class AsyncContextTool

Protocol defining an async tool that can be used by LLMs with context.

An AsyncContextTool represents an async function that can be called by an LLM during a call. It includes metadata like name, description, and parameter schema.

This class is not instantiated directly but created by the @tool() decorator.

Bases: ToolSchema[AsyncContextToolFn[DepsT, AnyP, JsonableCovariantT]], Generic[DepsT, JsonableCovariantT, AnyP]

Function execute

Execute the async context tool using an LLM-provided ToolCall.

Class AsyncContextToolkit

A collection of AsyncContextTools, with helpers for getting and executing specific tools.

Bases: BaseToolkit[AsyncTool | AsyncContextTool[DepsT]], Generic[DepsT]

Function execute

Execute an AsyncContextTool using the provided tool call.

Class AsyncResponse

The response generated by an LLM in async mode.

Bases:

Function execute_tools

Execute and return all of the tool calls in the response.

Function resume

Generate a new AsyncResponse using this response's messages with additional user content.

Uses this response's tools and format type. Also uses this response's provider, model, client, and params, unless the model context manager is being used to provide a new LLM as an override.

Attribute AsyncStream

Type: TypeAlias

An asynchronous assistant content stream.

Class AsyncStreamResponse

An AsyncStreamResponse wraps response content from the LLM with a streaming interface.

This class supports iteration to process chunks as they arrive from the model.

Content can be streamed in one of three ways:

• Via .streams(), which provides an iterator of streams, where each stream contains chunks of streamed data. The chunks contain deltas (new content in that particular chunk), and the stream itself accumulates the collected state of all the chunks processed thus far.
• Via .chunk_stream() which allows iterating over Mirascope's provider- agnostic chunk representation.
• Via .pretty_stream() a helper method which provides all response content as str deltas. Iterating through pretty_stream will yield text content and optionally placeholder representations for other content types, but it will still consume the full stream.
• Via .structured_stream(), a helper method which provides partial structured outputs from a response (useful when FormatT is set). Iterating through structured_stream will only yield structured partials, but it will still consume the full stream.

As chunks are consumed, they are collected in-memory on the AsyncContextStreamResponse, and they become available in .content, .messages, .tool_calls, etc. All of the stream iterators can be restarted after the stream has been consumed, in which case they will yield chunks from memory in the original sequence that came from the LLM. If the stream is only partially consumed, a fresh iterator will first iterate through in-memory content, and then will continue consuming fresh chunks from the LLM.

In the specific case of text chunks, they are included in the response content as soon as they become available, via an llm.Text part that updates as more deltas come in. This enables the behavior where resuming a partially-streamed response will include as much text as the model generated.

For other chunks, like Thinking or ToolCall, they are only added to response content once the corresponding part has fully streamed. This avoids issues like adding incomplete tool calls, or thinking blocks missing signatures, to the response.

For each iterator, fully iterating through the iterator will consume the whole LLM stream. You can pause stream execution midway by breaking out of the iterator, and you can safely resume execution from the same iterator if desired.

Bases:

Function execute_tools

Execute and return all of the tool calls in the response.

Function resume

Generate a new AsyncStreamResponse using this response's messages with additional user content.

Uses this response's tools and format type. Also uses this response's provider, model, client, and params, unless the model context manager is being used to provide a new LLM as an override.

Class AsyncTextStream

Asynchronous text stream implementation.

Bases:

Function collect

Asynchronously collect all chunks and return the final Text content.

Class AsyncThoughtStream

Asynchronous thought stream implementation.

Bases:

Function collect

Asynchronously collect all chunks and return the final Thought content.

Class AsyncTool

An async tool that can be used by LLMs.

An AsyncTool represents an async function that can be called by an LLM during a call. It includes metadata like name, description, and parameter schema.

This class is not instantiated directly but created by the @tool() decorator.

Bases: ToolSchema[AsyncToolFn[AnyP, JsonableCovariantT]], Generic[AnyP, JsonableCovariantT]

Function execute

Execute the async tool using an LLM-provided ToolCall.

Class AsyncToolCallStream

Asynchronous tool call stream implementation.

Bases:

Function collect

Asynchronously collect all chunks and return the final ToolCall content.

Class AsyncToolkit

A collection of AsyncTools, with helpers for getting and executing specific tools.

Bases:

Function execute

Execute an AsyncTool using the provided tool call.

Class Audio

Audio content for a message.

Audio can be included in messages for voice or sound-based interactions.

Function download

Download and encode an audio file from a URL.

Function download_async

Asynchronously download and encode an audio file from a URL.

Function from_file

Create an Audio from a file path.

Function from_bytes

Create an Audio from raw bytes.

Class AuthenticationError

Raised for authentication failures (401, invalid API keys).

Bases:

Class BadRequestError

Raised for malformed requests (400, 422).

Bases:

Class Base64AudioSource

Audio data represented as a base64 encoded string.

Class Base64ImageSource

Image data represented as a base64 encoded string.

Attribute ChunkIterator

Type: TypeAlias

Synchronous iterator yielding chunks with raw data.

Class ConnectionError

Raised when unable to connect to the API (network issues, timeouts).

Bases:

Class Context

Context for LLM calls.

This class provides a context for LLM calls, including the model, parameters, and any dependencies needed for the call.

Bases:

Class ContextResponse

The response generated by an LLM from a context call.

Bases: BaseResponse[ContextToolkit[DepsT], FormattableT], Generic[DepsT, FormattableT]

Function execute_tools

Execute and return all of the tool calls in the response.

Function resume

Generate a new ContextResponse using this response's messages with additional user content.

Uses this response's tools and format type. Also uses this response's provider, model, client, and params, unless the model context manager is being used to provide a new LLM as an override.

Class ContextStreamResponse

A ContextStreamResponse wraps response content from the LLM with a streaming interface.

This class supports iteration to process chunks as they arrive from the model.

Content can be streamed in one of three ways:

• Via .streams(), which provides an iterator of streams, where each stream contains chunks of streamed data. The chunks contain deltas (new content in that particular chunk), and the stream itself accumulates the collected state of all the chunks processed thus far.
• Via .chunk_stream() which allows iterating over Mirascope's provider- agnostic chunk representation.
• Via .pretty_stream() a helper method which provides all response content as str deltas. Iterating through pretty_stream will yield text content and optionally placeholder representations for other content types, but it will still consume the full stream.
• Via .structured_stream(), a helper method which provides partial structured outputs from a response (useful when FormatT is set). Iterating through structured_stream will only yield structured partials, but it will still consume the full stream.

As chunks are consumed, they are collected in-memory on the ContextStreamResponse, and they become available in .content, .messages, .tool_calls, etc. All of the stream iterators can be restarted after the stream has been consumed, in which case they will yield chunks from memory in the original sequence that came from the LLM. If the stream is only partially consumed, a fresh iterator will first iterate through in-memory content, and then will continue consuming fresh chunks from the LLM.

In the specific case of text chunks, they are included in the response content as soon as they become available, via an llm.Text part that updates as more deltas come in. This enables the behavior where resuming a partially-streamed response will include as much text as the model generated.

For other chunks, like Thinking or ToolCall, they are only added to response content once the corresponding part has fully streamed. This avoids issues like adding incomplete tool calls, or thinking blocks missing signatures, to the response.

For each iterator, fully iterating through the iterator will consume the whole LLM stream. You can pause stream execution midway by breaking out of the iterator, and you can safely resume execution from the same iterator if desired.

Bases: BaseSyncStreamResponse[ContextToolkit, FormattableT], Generic[DepsT, FormattableT]

Function execute_tools

Execute and return all of the tool calls in the response.

Function resume

Generate a new ContextStreamResponse using this response's messages with additional user content.

Uses this response's tools and format type. Also uses this response's provider, model, client, and params, unless the model context manager is being used to provide a new LLM as an override.

Class ContextTool

Protocol defining a tool that can be used by LLMs.

A ContextTool represents a function that can be called by an LLM during a call. It includes metadata like name, description, and parameter schema.

This class is not instantiated directly but created by the @tool() decorator.

Bases: ToolSchema[ContextToolFn[DepsT, AnyP, JsonableCovariantT]], Generic[DepsT, JsonableCovariantT, AnyP]

Function execute

Execute the context tool using an LLM-provided ToolCall.

Class ContextToolkit

A collection of ContextTools, with helpers for getting and executing specific tools.

Bases: BaseToolkit[Tool | ContextTool[DepsT]], Generic[DepsT]

Function execute

Execute a ContextTool using the provided tool call.

Class Document

Document content for a message.

Documents (like PDFs) can be included for the model to analyze or reference.

Function from_url

Create a Document from a URL.

Function from_file

Create a Document from a file path.

Function from_bytes

Create a Document from raw bytes.

Class FeatureNotSupportedError

Raised if a Mirascope feature is unsupported by chosen provider.

If compatibility is model-specific, then model_id should be specified. If the feature is not supported by the provider at all, then it may be None.

Bases:

Class FinishReason

The reason why the LLM finished generating a response.

FinishReason is only set when the response did not have a normal finish (e.g. it ran out of tokens). When a response finishes generating normally, no finish reason is set.

Bases: str, Enum

Class Format

Class representing a structured output format for LLM responses.

A Format contains metadata needed to describe a structured output type to the LLM, including the expected schema. This class is not instantiated directly, but is created by calling llm.format, or is automatically generated by LLM providers when a Formattable is passed to a call method.

Example:

from mirascope import llm

class Book:
    title: str
    author: str

print(llm.format(Book, mode="tool"))

Bases:

Attribute FormattingMode

Type: Literal['strict', 'json', 'tool']

Available modes for response format generation.

• "strict": Use strict mode for structured outputs, asking the LLM to strictly adhere to a given JSON schema. Not all providers or models support it, and may not be compatible with tool calling. When making a call using this mode, an llm.FormattingModeNotSupportedError error may be raised (if "strict" mode is wholly unsupported), or an llm.FeatureNotSupportedError may be raised (if trying to use strict along with tools and that is unsupported).

• "json": Use JSON mode for structured outputs. In contrast to strict mode, we ask the LLM to output JSON as text, though without guarantees that the model will output the expected format schema. If the provider has explicit JSON mode, it will be used; otherwise, Mirascope will modify the system prompt to request JSON output. May raise an llm.FeatureNotSupportedError if tools are present and the model does not support tool calling when using JSON mode.

• "tool": Use forced tool calling to structure outputs. Mirascope will construct an ad-hoc tool with the required json schema as tool args. When the LLM chooses that tool, it will automatically be converted from a ToolCall into regular response content (abstracting over the tool call). If other tools are present, they will be handled as regular tool calls.

Note: When llm.format is not used, the provider will automatically choose a mode at call time.

Class FormattingModeNotSupportedError

Raised when trying to use a formatting mode that is not supported by the chosen model.

Bases:

Class Image

Image content for a message.

Images can be included in messages to provide visual context. This can be used for both input (e.g., user uploading an image) and output (e.g., model generating an image).

Function from_url

Create an Image reference from a URL, without downloading it.

Function download

Download and encode an image from a URL.

Function download_async

Asynchronously download and encode an image from a URL.

Function from_file

Create an Image from a file path.

Function from_bytes

Create an Image from raw bytes.

Attribute Message

Type: TypeAlias

A message in an LLM interaction.

Messages have a role (system, user, or assistant) and content that is a sequence of content parts. The content can include text, images, audio, documents, and tool interactions.

For most use cases, prefer the convenience functions system(), user(), and assistant() instead of directly creating Message objects.

Example:

from mirascope import llm

messages = [
    llm.messages.system("You are a helpful assistant."),
    llm.messages.user("Hello, how are you?"),
]

Class MirascopeError

Base exception for all Mirascope errors.

Bases:

Class Model

The unified LLM interface that delegates to provider-specific clients.

This class provides a consistent interface for interacting with language models from various providers. It handles the common operations like generating responses, streaming, and async variants by delegating to the appropriate client methods.

Usage Note: In most cases, you should use llm.use_model() instead of instantiating Model directly. This preserves the ability to override the model at runtime using the llm.model() context manager. Only instantiate Model directly if you want to hardcode a specific model and prevent it from being overridden by context.

Example (recommended - allows override):

from mirascope import llm

def recommend_book(genre: str) -> llm.Response:
    # Uses context model if available, otherwise creates default
    model = llm.use_model(provider="openai", model_id="gpt-4o-mini")
    message = llm.messages.user(f"Please recommend a book in {genre}.")
    return model.call(messages=[message])

# Uses default model
response = recommend_book("fantasy")

# Override with different model
with llm.model(provider="anthropic", model_id="claude-sonnet-4-0"):
    response = recommend_book("fantasy")  # Uses Claude

Example (direct instantiation - prevents override):

from mirascope import llm

def recommend_book(genre: str) -> llm.Response:
    # Hardcoded model, cannot be overridden by context
    model = llm.Model(provider="openai", model_id="gpt-4o-mini")
    message = llm.messages.user(f"Please recommend a book in {genre}.")
    return model.call(messages=[message])

Function call

Generate an llm.Response by synchronously calling this model's LLM provider.

Function call_async

Generate an llm.AsyncResponse by asynchronously calling this model's LLM provider.

Function stream

Generate an llm.StreamResponse by synchronously streaming from this model's LLM provider.

Function stream_async

Generate an llm.AsyncStreamResponse by asynchronously streaming from this model's LLM provider.

Function context_call

Generate an llm.ContextResponse by synchronously calling this model's LLM provider.

Function context_call_async

Generate an llm.AsyncContextResponse by asynchronously calling this model's LLM provider.

Function context_stream

Generate an llm.ContextStreamResponse by synchronously streaming from this model's LLM provider.

Function context_stream_async

Generate an llm.AsyncContextStreamResponse by asynchronously streaming from this model's LLM provider.

Function resume

Generate a new llm.Response by extending another response's messages with additional user content.

Uses the previous response's tools and output format, and this model's params.

Depending on the client, this may be a wrapper around using client call methods with the response's messages and the new content, or it may use a provider-specific API for resuming an existing interaction.

Function resume_async

Generate a new llm.AsyncResponse by extending another response's messages with additional user content.

Uses the previous response's tools and output format, and this model's params.

Depending on the client, this may be a wrapper around using client call methods with the response's messages and the new content, or it may use a provider-specific API for resuming an existing interaction.

Function context_resume

Generate a new llm.ContextResponse by extending another response's messages with additional user content.

Uses the previous response's tools and output format, and this model's params.

Depending on the client, this may be a wrapper around using client call methods with the response's messages and the new content, or it may use a provider-specific API for resuming an existing interaction.

Function context_resume_async

Generate a new llm.AsyncContextResponse by extending another response's messages with additional user content.

Uses the previous response's tools and output format, and this model's params.

Depending on the client, this may be a wrapper around using client call methods with the response's messages and the new content, or it may use a provider-specific API for resuming an existing interaction.

Function resume_stream

Generate a new llm.StreamResponse by extending another response's messages with additional user content.

Uses the previous response's tools and output format, and this model's params.

Depending on the client, this may be a wrapper around using client call methods with the response's messages and the new content, or it may use a provider-specific API for resuming an existing interaction.

Function resume_stream_async

Generate a new llm.AsyncStreamResponse by extending another response's messages with additional user content.

Uses the previous response's tools and output format, and this model's params.

Depending on the client, this may be a wrapper around using client call methods with the response's messages and the new content, or it may use a provider-specific API for resuming an existing interaction.

Function context_resume_stream

Generate a new llm.ContextStreamResponse by extending another response's messages with additional user content.

Uses the previous response's tools and output format, and this model's params.

Depending on the client, this may be a wrapper around using client call methods with the response's messages and the new content, or it may use a provider-specific API for resuming an existing interaction.

Function context_resume_stream_async

Generate a new llm.AsyncContextStreamResponse by extending another response's messages with additional user content.

Uses the previous response's tools and output format, and this model's params.

Depending on the client, this may be a wrapper around using client call methods with the response's messages and the new content, or it may use a provider-specific API for resuming an existing interaction.

Class NotFoundError

Raised when requested resource is not found (404).

Bases:

Class Params

Common parameters shared across LLM providers.

Note: Each provider may handle these parameters differently or not support them at all. Please check provider-specific documentation for parameter support and behavior.

Bases:

Class Partial

Generate a new class with all attributes optionals.

Bases:

Class PermissionError

Raised for permission/authorization failures (403).

Bases:

Class RateLimitError

Raised when rate limits are exceeded (429).

Bases:

Class RawMessageChunk

A chunk containing provider-specific raw message content that will be added to the AssistantMessage.

This chunk contains a provider-specific representation of a piece of content that will be added to the AssistantMessage reconstructed by the containing stream. This content should be a Jsonable Python object for serialization purposes.

The intention is that this content may be passed as-is back to the provider when the generated AssistantMessage is being reused in conversation.

Class Response

The response generated by an LLM.

Bases:

Function execute_tools

Execute and return all of the tool calls in the response.

Function resume

Generate a new Response using this response's messages with additional user content.

Uses this response's tools and format type. Also uses this response's provider, model, client, and params, unless the model context manager is being used to provide a new LLM as an override.

Class ServerError

Raised for server-side errors (500+).

Bases:

Attribute Stream

Type: TypeAlias

A synchronous assistant content stream.

Class StreamResponse

A StreamResponse wraps response content from the LLM with a streaming interface.

This class supports iteration to process chunks as they arrive from the model.

Content can be streamed in one of three ways:

• Via .streams(), which provides an iterator of streams, where each stream contains chunks of streamed data. The chunks contain deltas (new content in that particular chunk), and the stream itself accumulates the collected state of all the chunks processed thus far.
• Via .chunk_stream() which allows iterating over Mirascope's provider- agnostic chunk representation.
• Via .pretty_stream() a helper method which provides all response content as str deltas. Iterating through pretty_stream will yield text content and optionally placeholder representations for other content types, but it will still consume the full stream.
• Via .structured_stream(), a helper method which provides partial structured outputs from a response (useful when FormatT is set). Iterating through structured_stream will only yield structured partials, but it will still consume the full stream.

As chunks are consumed, they are collected in-memory on the StreamResponse, and they become available in .content, .messages, .tool_calls, etc. All of the stream iterators can be restarted after the stream has been consumed, in which case they will yield chunks from memory in the original sequence that came from the LLM. If the stream is only partially consumed, a fresh iterator will first iterate through in-memory content, and then will continue consuming fresh chunks from the LLM.

In the specific case of text chunks, they are included in the response content as soon as they become available, via an llm.Text part that updates as more deltas come in. This enables the behavior where resuming a partially-streamed response will include as much text as the model generated.

For other chunks, like Thinking or ToolCall, they are only added to response content once the corresponding part has fully streamed. This avoids issues like adding incomplete tool calls, or thinking blocks missing signatures, to the response.

For each iterator, fully iterating through the iterator will consume the whole LLM stream. You can pause stream execution midway by breaking out of the iterator, and you can safely resume execution from the same iterator if desired.

Bases:

Function execute_tools

Execute and return all of the tool calls in the response.

Function resume

Generate a new StreamResponse using this response's messages with additional user content.

Uses this response's tools and format type. Also uses this response's provider, model, client, and params, unless the model context manager is being used to provide a new LLM as an override.

Attribute StreamResponseChunk

Type: TypeAlias

Attribute SystemContent

Type: TypeAlias

Type alias for content that can fit into a SystemMessage.

Class SystemMessage

A system message that sets context and instructions for the conversation.

Class Text

Text content for a message.

Class TextChunk

Represents an incremental text chunk in a stream.

Class TextEndChunk

Represents the end of a text chunk stream.

Class TextStartChunk

Represents the start of a text chunk stream.

Class TextStream

Synchronous text stream implementation.

Bases:

Function collect

Collect all chunks and return the final Text content.

Class Thought

Thinking content for a message.

Represents the thinking or thought process of the assistant. These generally are summaries of the model's reasoning process, rather than the direct reasoning tokens, although this behavior is model and provider specific.

Class ThoughtChunk

Represents an incremental thought chunk in a stream.

Class ThoughtEndChunk

Represents the end of a thought chunk stream.

Class ThoughtStartChunk

Represents the start of a thought chunk stream.

Class ThoughtStream

Synchronous thought stream implementation.

Bases:

Function collect

Collect all chunks and return the final Thought content.

Class TimeoutError

Raised when requests timeout or deadline exceeded.

Bases:

Class Tool

A tool that can be used by LLMs.

A Tool represents a function that can be called by an LLM during a call. It includes metadata like name, description, and parameter schema.

This class is not instantiated directly but created by the @tool() decorator.

Bases: ToolSchema[ToolFn[AnyP, JsonableCovariantT]], Generic[AnyP, JsonableCovariantT]

Function execute

Execute the tool using an LLM-provided ToolCall.

Class ToolCall

Tool call content for a message.

Represents a request from the assistant to call a tool. This is part of an assistant message's content.

Class ToolCallChunk

Represents an incremental tool call chunk in a stream.

Class ToolCallEndChunk

Represents the end of a tool call chunk stream.

Class ToolCallStartChunk

Represents the start of a tool call chunk stream.

Class ToolCallStream

Synchronous tool call stream implementation.

Bases:

Function collect

Collect all chunks and return the final ToolCall content.

Class ToolNotFoundError

Raised if a tool_call cannot be converted to any corresponding tool.

Bases:

Class ToolOutput

Tool output content for a message.

Represents the output from a tool call. This is part of a user message's content, typically following a tool call from the assistant.

Bases:

Class Toolkit

A collection of Tools, with helpers for getting and executing specific tools.

Bases:

Function execute

Execute a Tool using the provided tool call.

Class URLImageSource

Image data referenced via external URL.

Attribute UserContent

Type: TypeAlias

Type alias for content that can fit into a UserMessage.

Attribute UserContentPart

Type: TypeAlias

Content parts that can be included in a UserMessage.

Class UserMessage

A user message containing input from the user.

Function call

Returns a decorator for turning prompt template functions into generations.

This decorator creates a Call or ContextCall that can be used with prompt functions. If the first parameter is typed as llm.Context[T], it creates a ContextCall. Otherwise, it creates a regular Call.

Example:

Regular call:

from mirascope import llm

@llm.call(
    provider="openai:completions",
    model_id="gpt-4o-mini",
)
def answer_question(question: str) -> str:
    return f"Answer this question: {question}"

response: llm.Response = answer_question("What is the capital of France?")
print(response)

Example:

Context call:

from dataclasses import dataclass
from mirascope import llm

@dataclass
class Personality:
    vibe: str

@llm.call(
    provider="openai:completions",
    model_id="gpt-4o-mini",
)
def answer_question(ctx: llm.Context[Personality], question: str) -> str:
    return f"Your vibe is {ctx.deps.vibe}. Answer this question: {question}"

ctx = llm.Context(deps=Personality(vibe="snarky"))
response = answer_question(ctx, "What is the capital of France?")
print(response)

Module calls

The llm.calls module.

Function client

Create a cached client instance for the specified provider.

Module clients

Client interfaces for LLM providers.

Module content

The llm.messages.content module.

Module exceptions

Mirascope exception hierarchy for unified error handling across providers.

Function format

Returns a Format that describes structured output for a Formattable type.

This function converts a Formattable type (e.g. Pydantic BaseModel) into a Format object that describes how the object should be formatted. Calling llm.format is optional, as all the APIs that expect a Format can also take the Formattable type directly. However, calling llm.format is necessary in order to specify the formatting mode that will be used.

The Formattable type may provide custom formatting instructions via a formatting_instructions(cls) classmethod. If that method is present, it will be called, and the resulting instructions will automatically be appended to the system prompt.

If no formatting instructions are present, then Mirascope may auto-generate instructions based on the active format mode. To disable this behavior and all prompt modification, you can add the formatting_instructions classmethod and have it return None.

Module formatting

Response formatting interfaces for structuring LLM outputs.

This module provides a way to define structured output formats for LLM responses. The @format decorator can be applied to classes to specify how LLM outputs should be structured and parsed.

Function get_client

Get a client instance for the specified provider.

Multiple calls to get_client will return the same Client rather than constructing new ones.

Module mcp

MCP compatibility module.

Module messages

The messages module for LLM interactions.

This module defines the message types used in LLM interactions. Messages are represented as a unified Message class with different roles (system, user, assistant) and flexible content arrays that can include text, images, audio, documents, and tool interactions.

Function model

Set a model in context for the duration of the context manager.

This context manager sets a model that will be used by llm.use_model() calls within the context. This allows you to override the default model at runtime.

Example:

import mirascope.llm as llm

def recommend_book(genre: str) -> llm.Response:
    model = llm.use_model(provider="openai", model_id="gpt-4o-mini")
    message = llm.messages.user(f"Please recommend a book in {genre}.")
    return model.call(messages=[message])

# Override the default model at runtime
with llm.model(provider="anthropic", model_id="claude-sonnet-4-0"):
    response = recommend_book("fantasy")  # Uses Claude instead of GPT

Module models

The llm.models module for implementing the Model interface and utilities.

This module provides a unified interface for interacting with different LLM models through the Model class. The llm.model() context manager allows you to override the model at runtime, and llm.use_model() retrieves the model from context or creates a default one.

Function prompt

Prompt decorator for turning functions (or "Prompts") into prompts.

This decorator transforms a function into a Prompt, i.e. a function that returns list[llm.Message]. Its behavior depends on whether it's called with a spec string.

If the first parameter is named 'ctx' or typed as llm.Context[T], it creates a ContextPrompt. Otherwise, it creates a regular Prompt.

With a template string, it returns a PromptTemplateDecorator, in which case it uses the provided template to decorate an function with an empty body, and uses arguments to the function for variable substitution in the template. The resulting PromptTemplate returns messages based on the template.

Without a template string, it returns a PromptFunctionalDecorator, which transforms a Prompt (a function returning either message content, or messages) into a PromptTemplate. The resulting prompt template either promotes the content into a list containing a single user message, or passes along the messages returned by the decorated function.

Spec substitution rules:

• [USER], [ASSISTANT], [SYSTEM] demarcate the start of a new message with that role
• [MESSAGES] indicates the next variable contains a list of messages to include
• {{ variable }} injects the variable as a string, unless annotated
• Annotations: {{ variable:annotation }} where annotation is one of: image, images, audio, audios, document, documents
• Single content annotations (image, audio, document) expect a file path, URL, base64 string, or bytes, which becomes a content part with inferred mime-type
• Multiple content annotations (images, audios, documents) expect a list of strings or bytes, each becoming a content part with inferred mime-type

Module prompts

The prompt templates module for LLM interactions.

This module defines the prompt templates used in LLM interactions, which are written as python functions.

Module responses

The Responses module for LLM responses.

Function tool

Decorator that turns a function into a tool definition.

This decorator creates a Tool or ContextTool that can be used with llm.call. The function's name, docstring, and type hints are used to generate the tool's metadata.

If the first parameter is named 'ctx' or typed as llm.Context[T], it creates a ContextTool. Otherwise, it creates a regular Tool.

Examples:

Regular tool:

from mirascope import llm

@llm.tool
def available_books() -> list[str]:
    """Returns the list of available books."""
    return ["The Name of the Wind"]

Context tool:

from dataclasses import dataclass

from mirascope import llm


@dataclass
class Library:
    books: list[str]


library = Library(books=["Mistborn", "Gödel, Escher, Bach", "Dune"])

@llm.tool
def available_books(ctx: llm.Context[Library]) -> list[str]:
    """Returns the list of available books."""
    return ctx.deps.books

Module tools

The Tools module for LLMs.

Module types

Types for the LLM module.

Function use_model

Get the model from context if available, otherwise create a new Model.

This function checks if a model has been set in the context (via llm.model() context manager). If a model is found in the context, it returns that model. Otherwise, it creates and returns a new llm.Model instance with the provided arguments as defaults.

This allows you to write functions that work with a default model but can be overridden at runtime using the llm.model() context manager.

Example:

import mirascope.llm as llm

def recommend_book(genre: str) -> llm.Response:
    model = llm.use_model(provider="openai", model_id="gpt-4o-mini")
    message = llm.messages.user(f"Please recommend a book in {genre}.")
    return model.call(messages=[message])

# Uses the default model (gpt-4o-mini)
response = recommend_book("fantasy")

# Override with a different model
with llm.model(provider="anthropic", model_id="claude-sonnet-4-0"):
    response = recommend_book("fantasy")  # Uses Claude instead