# Reliability LLM API calls can fail for many reasons: rate limits, server errors, network issues, or malformed responses. Building reliable applications means handling these failures gracefully through retries, error recovery, and fallbacks. <Note> The Mirascope team is working on a Mirascope-native retry interface, to release shortly after Mirascope 2.0. </Note> ## Retrying Calls Wrap your calls in a retry loop to handle transient failures: ```python from mirascope import llm @llm.call("openai/gpt-4o-mini") def recommend_book(genre: str): return f"Recommend a {genre} book." max_retries = 3 for attempt in range(max_retries): try: response = recommend_book("fantasy") print(response.pretty()) break except llm.Error: if attempt == max_retries - 1: raise ``` This retries up to 3 times before re-raising the error. ### Retrying Specific Errors Not all errors should be retried. Authentication errors won't succeed on retry, but rate limits and server errors often will: ```python from mirascope import llm @llm.call("openai/gpt-4o-mini") def recommend_book(genre: str): return f"Recommend a {genre} book." max_retries = 3 for attempt in range(max_retries): try: response = recommend_book("fantasy") print(response.pretty()) break except (llm.RateLimitError, llm.ServerError): # Retry on rate limits and server errors if attempt == max_retries - 1: raise except llm.AuthenticationError: # Don't retry auth errors - they won't succeed raise ``` Mirascope provides [unified error types](/docs/learn/llm/errors) that work across all providers, so you can write provider-agnostic retry logic. ## Retrying Streams Streaming responses can fail mid-stream—the initial connection succeeds, but an error occurs while iterating. Wrap the streaming loop and use `response.resume()` to continue from where you left off: ```python from mirascope import llm @llm.call("openai/gpt-4o-mini") def recommend_book(genre: str): return f"Recommend a {genre} book." max_retries = 3 response = recommend_book.stream("fantasy") for attempt in range(max_retries): try: for chunk in response.text_stream(): print(chunk, end="", flush=True) break except llm.Error: if attempt == max_retries - 1: raise # Stream failed mid-way. Resume from where we left off. response = response.resume("Please continue.") ``` This works because `StreamResponse` accumulates content as you iterate. When an error occurs, `response.messages` contains everything received up to that point, so the model has context about what it already said. ## Retrying on Structured Output Parse Errors When using [structured output](/docs/structured-output), the LLM's response might fail to parse—either because the JSON is malformed or because it doesn't match the expected schema. Mirascope wraps all parsing failures in `llm.ParseError`, which provides the original exception. Instead of retrying blindly, use `error.retry_message()` to tell the model what went wrong so it can fix its mistake: ```python from pydantic import BaseModel from mirascope import llm class Book(BaseModel): title: str author: str year: int @llm.call("openai/gpt-4o-mini", format=Book) def recommend_book(genre: str): return f"Recommend a {genre} book." max_retries = 3 response = recommend_book("fantasy") for attempt in range(max_retries): try: book = response.parse() print(f"{book.title} by {book.author} ({book.year})") break except llm.ParseError as e: if attempt == max_retries - 1: raise # Tell the model what went wrong so it can fix it response = response.resume(e.retry_message()) ``` This pattern—catching the error and resuming with feedback—gives the model a chance to self-correct. It's more effective than blind retries because the model learns from its mistake. The `ParseError` wraps several underlying error types: - `ValueError` when no valid JSON object is found in the response - `json.JSONDecodeError` when the JSON syntax is invalid - `pydantic.ValidationError` when the JSON doesn't match the schema - Any exception from custom [OutputParsers](/docs/structured-output#custom-parsing) ## Tool Error Recovery Tools can fail during execution. Mirascope automatically catches these errors and includes them in the `ToolOutput`, so the error message is passed to the LLM and it can adapt—no manual exception handling required. If you want to monitor for errors or limit retries, check `tool_output.error`: ```python from mirascope import llm @llm.tool def divide(a: float, b: float) -> float: """Divide a by b.""" return a / b @llm.call("openai/gpt-4o-mini", tools=[divide]) def calculator(query: str): return query response = calculator("What is 10 divided by 0?") # Mirascope automatically catches tool execution errors and includes them # in the ToolOutput. The error message is passed to the LLM so it can adapt. max_retries = 3 consecutive_errors = 0 while response.tool_calls: tool_outputs = response.execute_tools() # Check if any tools failed had_errors = any(output.error is not None for output in tool_outputs) if had_errors: consecutive_errors += 1 if consecutive_errors >= max_retries: # Find the first error and raise it for output in tool_outputs: if output.error is not None: raise output.error else: consecutive_errors = 0 # Resume with tool outputs - errors are automatically passed to the LLM response = response.resume(tool_outputs) print(response.text()) ``` The model receives the error message and can explain what went wrong or try a different approach. If the same error keeps occurring, you can raise it after a set number of retries. ## Provider Fallbacks Combine retries with fallbacks: retry each provider multiple times before moving to the next: <TabbedSection> <Tab value="Call"> ```python from mirascope import llm models = [ "openai/gpt-4o-mini", "anthropic/claude-3-5-haiku-latest", "google/gemini-2.0-flash", ] @llm.call(models[0]) def recommend_book(genre: str): return f"Recommend a {genre} book." def with_fallbacks(genre: str, max_retries: int = 3) -> llm.Response: errors: list[llm.Error] = [] for model_id in models: for attempt in range(max_retries): try: with llm.model(model_id): return recommend_book(genre) except llm.Error as e: errors.append(e) if attempt == max_retries - 1: break # Try next model # Re-raise last error (simple approach) raise errors[-1] response = with_fallbacks("fantasy") print(response.pretty()) ``` </Tab> <Tab value="Prompt"> ```python from mirascope import llm models = [ "openai/gpt-4o-mini", "anthropic/claude-3-5-haiku-latest", "google/gemini-2.0-flash", ] @llm.prompt def recommend_book(genre: str): return f"Recommend a {genre} book." def with_fallbacks(genre: str, max_retries: int = 3) -> llm.Response: errors: list[llm.Error] = [] for model_id in models: for attempt in range(max_retries): try: return recommend_book(model_id, genre) except llm.Error as e: errors.append(e) if attempt == max_retries - 1: break # Try next model # Re-raise last error (simple approach) raise errors[-1] response = with_fallbacks("fantasy") print(response.pretty()) ``` </Tab> <Tab value="Model"> ```python from mirascope import llm models = [ "openai/gpt-4o-mini", "anthropic/claude-3-5-haiku-latest", "google/gemini-2.0-flash", ] def recommend_book(model: llm.Model, genre: str): return model.call(f"Recommend a {genre} book.") def with_fallbacks(genre: str, max_retries: int = 3) -> llm.Response: errors: list[llm.Error] = [] for model_id in models: model = llm.model(model_id) for attempt in range(max_retries): try: return recommend_book(model, genre) except llm.Error as e: errors.append(e) if attempt == max_retries - 1: break # Try next model # Re-raise last error (simple approach) raise errors[-1] response = with_fallbacks("fantasy") print(response.pretty()) ``` </Tab> </TabbedSection> ## Next Steps - [Errors](/docs/learn/llm/errors) — Unified error types across providers - [Streaming](/docs/learn/llm/streaming) — Streaming patterns in depth - [Structured Output](/docs/learn/llm/structured-output) — Validation and parsing