{ "cells": [ { "cell_type": "markdown", "id": "545b05ef21c691cb", "metadata": {}, "source": [ "# Self-Refine: Enhancing LLM Outputs Through Iterative Self-Improvement\n", "\n", "This recipe demonstrates how to implement the Self-Refine technique using Large Language Models (LLMs) with Mirascope. Self-Refine is a prompt engineering method that enhances an LLM's output by iteratively generating feedback and improving its responses.\n", "\n", "
\n", "

Mirascope Concepts Used

\n", "\n", "
\n", "\n", "
\n", "

Background

\n", "

\n", "Self-refine is a prompt engineering technique where a model gives feedback about its answer and uses the feedback to generate a new answer. This self refinement can take place multiple times to generate the final answer. Self-refine is helpful for reasoning, coding, and generation tasks.\n", "

\n", "
\n", "\n", "## Basic Self-Refine Implementation\n", "\n", "Let's start with a basic implementation of Self-Refine using Mirascope:\n", "\n" ] }, { "cell_type": "code", "execution_count": 1, "id": "7c704009c03faa79", "metadata": { "ExecuteTime": { "end_time": "2024-10-01T02:34:37.265691Z", "start_time": "2024-10-01T02:34:29.603167Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "To determine how much money Olivia has left after her purchase, let's break it down step by step:\n", "\n", "1. **Starting Amount**: Olivia has $23 initially.\n", "2. **Cost of Bagels**: She bought 5 bagels at $3 each. The total spent on bagels is calculated as:\n", " \\[\n", " 5 \\times 3 = 15 \\text{ dollars}\n", " \\]\n", "3. **Amount Left**: Now, we subtract the total amount spent on the bagels from Olivia's starting amount:\n", " \\[\n", " 23 - 15 = 8 \\text{ dollars}\n", " \\]\n", "\n", "Therefore, after buying the bagels, Olivia has **$8 remaining**.\n" ] } ], "source": [ "from mirascope.core import openai, prompt_template\n", "from mirascope.core.openai import OpenAICallResponse\n", "\n", "\n", "@openai.call(model=\"gpt-4o-mini\")\n", "def call(query: str) -> str:\n", " return query\n", "\n", "\n", "@openai.call(model=\"gpt-4o-mini\")\n", "@prompt_template(\n", " \"\"\"\n", " Here is a query and a response to the query. Give feedback about the answer,\n", " noting what was correct and incorrect.\n", " Query:\n", " {query}\n", " Response:\n", " {response}\n", " \"\"\"\n", ")\n", "def evaluate_response(query: str, response: OpenAICallResponse): ...\n", "\n", "\n", "@openai.call(model=\"gpt-4o-mini\")\n", "@prompt_template(\n", " \"\"\"\n", " For this query:\n", " {query}\n", " The following response was given:\n", " {response}\n", " Here is some feedback about the response:\n", " {feedback}\n", "\n", " Consider the feedback to generate a new response to the query.\n", " \"\"\"\n", ")\n", "def generate_new_response(\n", " query: str, response: OpenAICallResponse\n", ") -> openai.OpenAIDynamicConfig:\n", " feedback = evaluate_response(query, response)\n", " return {\"computed_fields\": {\"feedback\": feedback}}\n", "\n", "\n", "def self_refine(query: str, depth: int) -> str:\n", " response = call(query)\n", " for _ in range(depth):\n", " response = generate_new_response(query, response)\n", " return response.content\n", "\n", "\n", "query = \"\"\"Olivia has $23. She bought five bagels for $3 each.\n", "How much money does she have left?\"\"\"\n", "print(self_refine(query, 1))" ] }, { "cell_type": "markdown", "id": "79c07fd5020db853", "metadata": {}, "source": [ "## Enhanced Self-Refine with Response Model\n", "\n", "Now, let's improve our implementation by adding a response model to structure the output:\n" ] }, { "cell_type": "code", "execution_count": 2, "id": "ec5b18083d5020b5", "metadata": { "ExecuteTime": { "end_time": "2024-10-01T02:35:03.257847Z", "start_time": "2024-10-01T02:34:57.003576Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "steps=['Olivia has $23.', 'She bought five bagels for $3 each.', 'Calculate the total cost for the bagels: 5 bagels * $3/bagel = $15.', 'Subtract the total cost of the bagels from the amount of money she had: $23 - $15 = $8.'] final_answer=8.0\n" ] } ], "source": [ "from pydantic import BaseModel, Field\n", "\n", "\n", "class MathSolution(BaseModel):\n", " steps: list[str] = Field(..., description=\"The steps taken to solve the problem\")\n", " final_answer: float = Field(..., description=\"The final numerical answer\")\n", "\n", "\n", "@openai.call(model=\"gpt-4o-mini\", response_model=MathSolution)\n", "@prompt_template(\n", " \"\"\"\n", " For this query:\n", " {query}\n", " The following response was given:\n", " {response}\n", " Here is some feedback about the response:\n", " {feedback}\n", "\n", " Consider the feedback to generate a new response to the query.\n", " Provide the solution steps and the final numerical answer.\n", " \"\"\"\n", ")\n", "def enhanced_generate_new_response(\n", " query: str, response: OpenAICallResponse\n", ") -> openai.OpenAIDynamicConfig:\n", " feedback = evaluate_response(query, response)\n", " return {\"computed_fields\": {\"feedback\": feedback}}\n", "\n", "\n", "def enhanced_self_refine(query: str, depth: int) -> MathSolution:\n", " response = call(query)\n", " for _ in range(depth):\n", " solution = enhanced_generate_new_response(query, response)\n", " response = f\"Steps: {solution.steps}\\nFinal Answer: {solution.final_answer}\"\n", " return solution\n", "\n", "\n", "# Example usage\n", "result = enhanced_self_refine(query, 1)\n", "print(result)" ] }, { "cell_type": "markdown", "id": "f8c4d8679d6baa3", "metadata": {}, "source": [ "This enhanced version introduces a MathSolution response model to structure the output, providing a clearer separation between solution steps and the final answer.\n", "\n", "## Benefits and Considerations\n", "\n", "The Self-Refine implementation offers several advantages:\n", "\n", "1. Improved accuracy through iterative refinement of responses.\n", "2. Enhanced reasoning capabilities, especially for complex problems.\n", "3. Potential for generating more detailed and step-by-step solutions.\n", "\n", "When implementing this technique, consider:\n", "\n", "- Balancing the number of refinement iterations with computational cost and time constraints.\n", "- Tailoring the feedback prompts to focus on specific aspects of improvement relevant to your use case.\n", "- Experimenting with different model parameters (e.g., temperature) for initial responses vs. refinement steps.\n", "\n", "
\n", "

Additional Real-World Applications

\n", "\n", "
\n", "\n", "When adapting this recipe to your specific use-case, consider:\n", "\n", "- Customizing the feedback prompts to focus on domain-specific criteria.\n", "- Implementing different types of response models for various tasks (e.g., text generation, problem-solving).\n", "- Combining Self-Refine with other techniques like Chain of Thought for more complex reasoning tasks.\n", "- Developing a mechanism to halt refinement when improvements become marginal.\n", "\n", "By leveraging Mirascope's call decorator, response models, and dynamic configuration, you can easily implement and customize the Self-Refine technique to enhance your LLM's output quality across a wide range of applications." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.6" } }, "nbformat": 4, "nbformat_minor": 5 }