{ "cells": [ { "metadata": {}, "cell_type": "markdown", "source": [ "# Decomposed Prompting: Enhancing LLM Problem-Solving with Tool-Based Subproblems\n", "\n", "This recipe demonstrates how to implement the Decomposed Prompting (DECOMP) technique using Large Language Models (LLMs) with Mirascope. DECOMP is a prompt engineering method that enhances an LLM's problem-solving capabilities by breaking down complex problems into subproblems and utilizing tools to solve each step.\n", "\n", "
\n", "

Mirascope Concepts Used

\n", "\n", "
\n", "\n", "
\n", "

Background

\n", "

\n", "Decomposed Prompting (DECOMP) is an extension of least-to-most whereby tools are used to execute each subproblem in the problem solving process. A pre-trained call (in our case, a one shot prompt) demonstrates how to break a problem down into subproblems within the context of its given tool calls, and the output of each tool call is added to the chat's history until the problem is solved. Just like least-to-most, DECOMP shows improvements on mathematical reasoning and symbolic manipulation tasks, with better results than least-to-most.\n", "

\n", "
\n", "\n", "## Implementation\n", "\n", "Let's implement the Decomposed Prompting technique using Mirascope:\n", "\n" ], "id": "3fd6603340953701" }, { "metadata": { "ExecuteTime": { "end_time": "2024-10-01T01:24:01.960870Z", "start_time": "2024-10-01T01:23:49.523104Z" } }, "cell_type": "code", "source": [ "import json\n", "\n", "from mirascope.core import openai, prompt_template\n", "from openai.types.chat import ChatCompletionMessageParam\n", "from pydantic import BaseModel, Field\n", "\n", "\n", "class Problem(BaseModel):\n", " subproblems: list[str] = Field(\n", " ..., description=\"The subproblems that the original problem breaks down into\"\n", " )\n", "\n", "\n", "@openai.call(model=\"gpt-4o\", response_model=Problem)\n", "@prompt_template(\n", " \"\"\"\n", " Your job is to break a problem into subproblems so that it may be solved\n", " step by step, using at most one function call at each step.\n", "\n", " You have access to the following functions which you can use to solve a\n", " problem:\n", " split: split a string into individual words\n", " substring: get the ith character of a single string.\n", " concat: concatenate some number of strings.\n", "\n", " Here is an example of how it would be done for the problem: Get the first two\n", " letters of the phrase 'Round Robin' with a period and space in between them.\n", " Steps:\n", " split 'Round Robin' into individual words\n", " substring the 0th char of 'Round'\n", " substring the 0th char of 'Robin'\n", " concat ['R', '.', ' ', 'R']\n", "\n", " Now, turn this problem into subtasks:\n", " {query}\n", " \"\"\"\n", ")\n", "def break_into_subproblems(query: str): ...\n", "\n", "\n", "def split_string_to_words(string: str) -> str:\n", " \"\"\"Splits a string into words.\"\"\"\n", " return json.dumps(string.split())\n", "\n", "\n", "def substring(index: int, string: str) -> str:\n", " \"\"\"Gets the character at the index of a string.\"\"\"\n", " return string[index]\n", "\n", "\n", "def concat(strings: list[str]) -> str:\n", " \"\"\"Concatenates some number of strings.\"\"\"\n", " return \"\".join(strings)\n", "\n", "\n", "@openai.call(model=\"gpt-4o-mini\", tools=[split_string_to_words, substring, concat])\n", "@prompt_template(\n", " \"\"\"\n", " SYSTEM: You are being fed subproblems to solve the actual problem: {query}\n", " MESSAGES: {history}\n", " \"\"\"\n", ")\n", "def solve_next_step(history: list[ChatCompletionMessageParam], query: str): ...\n", "\n", "\n", "def decomposed_prompting(query: str):\n", " problem = break_into_subproblems(query=query)\n", " response = None\n", " history: list[ChatCompletionMessageParam] = []\n", " for subproblem in problem.subproblems:\n", " history.append({\"role\": \"user\", \"content\": subproblem})\n", " response = solve_next_step(history, query)\n", " history.append(response.message_param)\n", " if tool := response.tool:\n", " output = tool.call()\n", " history += response.tool_message_params([(tool, output)])\n", " response = solve_next_step(history, query)\n", "\n", " # This should never return another tool call in DECOMP so don't recurse\n", " history.append(response.message_param)\n", " return response\n", "\n", "\n", "query = \"\"\"Take the last letters of the words in \"Augusta Ada King\" and concatenate them using a space.\"\"\"\n", "\n", "\n", "print(decomposed_prompting(query))" ], "id": "d0dbfe29816a392", "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The concatenated characters are \"aag\".\n" ] } ], "execution_count": 1 }, { "metadata": {}, "cell_type": "markdown", "source": [ "This implementation consists of several key components:\n", "\n", "1. `Problem` class: Defines the structure for breaking down a problem into subproblems.\n", "2. `break_into_subproblems`: Uses GPT-4o-mini to break the main problem into subproblems.\n", "3. Tool functions: `split`, `substring`, and `concat` for manipulating strings.\n", "4. `solve_next_step`: Uses GPT-3.5-turbo to solve each subproblem, utilizing the available tools.\n", "5. `decomposed_prompting`: Orchestrates the entire process, solving subproblems sequentially and maintaining conversation history.\n", "\n", "## Benefits and Considerations\n", "\n", "The Decomposed Prompting implementation offers several advantages:\n", "\n", "1. Improved problem-solving capabilities for complex tasks.\n", "2. Better handling of multi-step problems that require different operations.\n", "3. Increased transparency in the problem-solving process.\n", "4. Potential for solving problems that are beyond the scope of a single LLM call.\n", "\n", "When implementing this technique, consider:\n", "\n", "- Carefully designing the set of available tools to cover a wide range of problem-solving needs.\n", "- Balancing the complexity of subproblems with the capabilities of the chosen LLM.\n", "- Implementing error handling and recovery mechanisms for cases where a subproblem solution fails.\n", "- Optimizing the prompt for breaking down problems to ensure effective decomposition.\n", "\n", "
\n", "

Additional Real-World Applications

\n", "\n", "
\n", "\n", "When adapting this recipe to your specific use-case, consider:\n", "\n", "- Tailoring the available tools to your domain for better performance.\n", "- Implementing a feedback loop to refine the problem decomposition process based on solution accuracy.\n", "- Combining Decomposed Prompting with other techniques like Chain of Thought for even more powerful problem-solving capabilities.\n", "- Developing a mechanism to handle interdependencies between subproblems.\n", "\n", "By leveraging Mirascope's `call` decorator, response models, and dynamic configuration, you can easily implement and customize the Decomposed Prompting technique to enhance your LLM's problem-solving capabilities across a wide range of applications." ], "id": "8c6ccd6b5b624150" } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.6" } }, "nbformat": 4, "nbformat_minor": 5 }