{ "cells": [ { "cell_type": "markdown", "id": "3628bd00fa153674", "metadata": {}, "source": [ "# Least to Most: Enhancing LLM Reasoning with Subproblem Decomposition\n", "\n", "This recipe demonstrates how to implement the Least to Most technique using Large Language Models (LLMs) with Mirascope. Least to Most is a prompt engineering method that enhances an LLM's reasoning capabilities by breaking down complex problems into smaller, more manageable subproblems.\n", "\n", "
\n", "

Mirascope Concepts Used

\n", "\n", "
\n", "\n", "
\n", "

Background

\n", "

\n", "Least to Most is a more extensive version of Chain of Thought, where separate calls are used to break down the original problem into subproblems as well as solve each individual step/subproblem. After solving each subproblem, the result is appended to the chat's history until the original problem is solved. Least to Most is an effective technique for symbolic and arithmetic reasoning tasks.\n", "

\n", "
\n", "\n", "## Implementing Least to Most\n", "\n", "Let's implement the Least to Most technique using Mirascope:\n" ] }, { "cell_type": "code", "execution_count": 1, "id": "786cbdbe53cb66d", "metadata": { "ExecuteTime": { "end_time": "2024-10-01T01:51:53.472071Z", "start_time": "2024-10-01T01:51:47.344562Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "To calculate the duration between two years, you subtract the earlier year from the later year. The formula is:\n", "\n", "\\[ \\text{Duration} = \\text{Later Year} - \\text{Earlier Year} \\]\n", "\n", "For example, if you want to calculate the duration between 2005 and 2010:\n", "\n", "\\[ \\text{Duration} = 2010 - 2005 = 5 \\]\n", "\n", "So, the duration between 2005 and 2010 is 5 years.\n" ] } ], "source": [ "from mirascope.core import openai, prompt_template\n", "from mirascope.core.openai import OpenAICallResponse\n", "from openai.types.chat import ChatCompletionMessageParam\n", "from pydantic import BaseModel, Field\n", "\n", "few_shot_examples = [\n", " {\n", " \"question\": \"The median age in the city was 22.1 years. 10.1% of residents were under the age of 18; 56.2% were between the ages of 18 and 24; 16.1% were from 25 to 44; 10.5% were from 45 to 64; and 7% were 65 years of age or older. Which age group is larger: under the age of 18 or 18 and 24?\",\n", " \"answer\": 'To answer the question \"Which age group is larger: under the age of 18 or 18 and 24?\", we need to know: \"How many percent were under the age of 18?\", \"How many percent were between the ages of 18 and 24?\".',\n", " },\n", " {\n", " \"question\": \"Old age pensions were raised by 300 francs per month to 1,700 francs for a single person and to 3,700 francs for a couple, while health insurance benefits were made more widely available to unemployed persons and part-time employees. How many francs were the old age pensions for a single person before they were raised?\",\n", " \"answer\": 'To answer the question \"How many francs were the old age pensions for a single person before they were raised?\", we need to know: \"How many francs were the old age pensions for a single person?\", \"How many francs were old age pensions raised for a single person?\".',\n", " },\n", " {\n", " \"question\": \"In April 2011, the ECB raised interest rates for the first time since 2008 from 1% to 1.25%, with a further increase to 1.50% in July 2011. However, in 2012-2013 the ECB lowered interest rates to encourage economic growth, reaching the historically low 0.25% in November 2013. Soon after the rates were cut to 0.15%, then on 4 September 2014 the central bank reduced the rates from 0.15% to 0.05%, the lowest rates on record. How many percentage points did interest rates drop between April 2011 and September 2014?\",\n", " \"answer\": 'To answer the question \"How many percentage points did interest rates drop between April 2011 and September 2014?\", we need to know: \"What was the interest rate in April 2011?\", \"What was the interest rate in September 2014?\".',\n", " },\n", " {\n", " \"question\": \"Non-nationals make up more than half of the population of Bahrain. According to government statistics dated between 2005-2009 roughly 290,000 Indians, 125,000 Bangladeshis, 45,000 Pakistanis, 45,000 Filipinos, and 8,000 Indonesians. How many Pakistanis and Indonesians are in Bahrain?\",\n", " \"answer\": 'To answer the question \"How many Pakistanis and Indonesians are in Bahrain?\", we need to know: \"How many Pakistanis are in Bahrain?\", \"How many Indonesians are in Bahrain?\".',\n", " },\n", "]\n", "\n", "\n", "class Problem(BaseModel):\n", " subproblems: list[str] = Field(\n", " ..., description=\"The subproblems that the original problem breaks down into\"\n", " )\n", "\n", "\n", "@openai.call(model=\"gpt-4o-mini\", response_model=Problem)\n", "@prompt_template(\n", " \"\"\"\n", " Examples to guide your answer:\n", " {examples:lists}\n", " Break the following query into subproblems:\n", " {query}\n", " \"\"\"\n", ")\n", "def break_into_subproblems(\n", " query: str, few_shot_examples: list[dict[str, str]]\n", ") -> openai.OpenAIDynamicConfig:\n", " examples = [\n", " [f\"Q:{example['question']}\", f\"A:{example['answer']}\"]\n", " for example in few_shot_examples\n", " ]\n", " return {\"computed_fields\": {\"examples\": examples}}\n", "\n", "\n", "@openai.call(model=\"gpt-4o-mini\")\n", "def call(history: list[ChatCompletionMessageParam]) -> str:\n", " return f\"MESSAGES: {history}\"\n", "\n", "\n", "def least_to_most(query_context: str, query_question: str) -> OpenAICallResponse:\n", " problem = break_into_subproblems(\n", " query=query_context + query_question, few_shot_examples=few_shot_examples\n", " )\n", " history: list[ChatCompletionMessageParam] = [\n", " {\"role\": \"user\", \"content\": query_context + problem.subproblems[0]}\n", " ]\n", " response = call(history=history)\n", " history.append(response.message_param)\n", " if len(problem.subproblems) == 1:\n", " return response\n", " else:\n", " for i in range(1, len(problem.subproblems)):\n", " history.append({\"role\": \"user\", \"content\": problem.subproblems[i]})\n", " response = call(history=history)\n", " history.append(response.message_param)\n", " return response\n", "\n", "\n", "query_context = \"\"\"The Census Bureaus 2006-2010 American Community Survey showed that \\\n", "(in 2010 inflation adjustment dollars) median household income was $52,056 and the \\\n", "median family income was $58,942.\"\"\"\n", "\n", "query_question = \"How many years did the Census Bureaus American Community Survey last?\"\n", "\n", "print(least_to_most(query_context=query_context, query_question=query_question))" ] }, { "cell_type": "markdown", "id": "69d7b61e65481ad9", "metadata": {}, "source": [ "This implementation consists of three main components:\n", "\n", "1. `break_into_subproblems`: This function takes a query and breaks it down into subproblems using few-shot examples.\n", "2. `call`: A simple function that makes a call to the LLM with the given message history.\n", "3. `least_to_most`: This function orchestrates the Least to Most technique. It first breaks the problem into subproblems, then solves each subproblem sequentially, appending the results to the message history.\n", "\n", "## Benefits and Considerations\n", "\n", "The Least to Most implementation offers several advantages:\n", "\n", "1. Improved reasoning for complex problems by breaking them down into manageable steps.\n", "2. Enhanced ability to handle multi-step arithmetic and symbolic reasoning tasks.\n", "3. Potential for more accurate responses by solving subproblems individually.\n", "\n", "When implementing this technique, consider:\n", "\n", "- Carefully crafting few-shot examples to guide the problem decomposition process.\n", "- Balancing the number of subproblems to avoid oversimplification or overcomplexity.\n", "- Ensuring that the query context and question are clear and contain all necessary information.\n", "\n", "
\n", "

Additional Real-World Applications

\n", "\n", "
\n", "\n", "When adapting this recipe to your specific use-case, consider:\n", "\n", "- Tailoring the few-shot examples to your domain for better problem decomposition.\n", "- Implementing a mechanism to handle interdependent subproblems.\n", "- Combining Least to Most with other techniques like Self-Consistency for even more robust reasoning.\n", "- Developing a feedback loop to refine the problem decomposition process based on the accuracy of final answers.\n", "\n", "By leveraging Mirascope's `call` decorator, response models, and dynamic configuration, you can easily implement and customize the Least to Most technique to enhance your LLM's ability to reason about complex problems across a wide range of applications." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.6" } }, "nbformat": 4, "nbformat_minor": 5 }