System to Attention (S2A): Enhancing LLM Focus with Query Filtering¶
This recipe demonstrates how to implement the System to Attention (S2A) technique using Large Language Models (LLMs) with Mirascope. S2A is a prompt engineering method that enhances an LLM's ability to focus on relevant information by filtering out irrelevant context from the initial query.
Mirascope Concepts Used
Background
System to Attention (S2A) is a prompt engineering technique whereby the prompt is first filtered to remove all irrelevant information from the query. This approach helps LLMs focus on the most pertinent information, potentially improving the accuracy and relevance of their responses, especially for queries containing extraneous or potentially biasing information.
Implementation¶
Let's implement the S2A technique using Mirascope:
from mirascope.core import openai, prompt_template
from pydantic import BaseModel, Field
class RelevantContext(BaseModel):
context_text: str = Field(
description="Context text related to the question (includes all content except unrelated sentences)"
)
detailed_question: str = Field(description="Detailed question:")
@openai.call(model="gpt-4o-mini", response_model=RelevantContext)
@prompt_template(
"""
Given the following text by a user, extract the part that is related and useful, so that using that text alone would be good context for providing an accurate and correct answer to the question portion of the text.
Please include the actual question or query that the user is asking.
Separate this into two categories labeled with ”Context text related to the question (includes all content except unrelated sentences):” and ”Detailed question:”.
Do not use list.
Text by User: {query}
"""
)
def remove_irrelevant_info(query: str):
"""Reduces a query down to its relevant context and question"""
@openai.call(model="gpt-4o-mini")
@prompt_template(
"""
Original user query (possibly biased): {query}
Unbiased context: {context_text}
Given the above unbiased context, answer the following: {detailed_question}
"""
)
def s2a(query: str) -> openai.OpenAIDynamicConfig:
"""Executes the flow of the System to Attention technique."""
relevant_context = remove_irrelevant_info(query=query)
context_text = relevant_context.context_text
detailed_question = relevant_context.detailed_question
return {
"computed_fields": {
"context_text": context_text,
"detailed_question": detailed_question,
}
}
# Example usage
query = """Sunnyvale is a city in California. \
Sunnyvale has many parks. Sunnyvale city is \
close to the mountains. Many notable people \
are born in Sunnyvale. \
In which city was San Jose's mayor Sam \
Liccardo born?"""
print(s2a(query=query))
Sam Liccardo, the mayor of San Jose, was born in San Jose, California.
This implementation consists of two main functions:
remove_irrelevant_info: This function takes the original query and extracts the relevant context and the detailed question. It uses aRelevantContextresponse model to structure the output.s2a: This is the main function that orchestrates the S2A technique. It first callsremove_irrelevant_infoto filter the query, then uses the filtered information to generate a response.
How It Works¶
Query Filtering: The
remove_irrelevant_infofunction analyzes the input query and separates it into relevant context and the actual question. This step helps remove any irrelevant or potentially biasing information.Context Separation: The filtered information is structured into two parts: the context text and the detailed question. This separation allows for more focused processing in the next step.
Unbiased Response Generation: The
s2afunction uses the filtered context and question to generate a response. By providing the original query alongside the filtered information, it allows the model to be aware of potential biases while focusing on the relevant information.
Benefits and Considerations¶
The S2A technique offers several advantages:
- Improved focus on relevant information, potentially leading to more accurate responses.
- Reduction of bias from irrelevant context in the original query.
- Clear separation of context and question, allowing for more structured reasoning.
When implementing this technique, consider:
- Balancing between removing irrelevant information and retaining important context.
- Adjusting the filtering prompt based on the specific domain or type of queries you're dealing with.
- Monitoring the performance to ensure that important information isn't being filtered out unintentionally.
Additional Real-World Applications
- Customer Support: Filter out emotional language or irrelevant details from customer queries to focus on the core issue.
- Legal Document Analysis: Extract relevant facts and questions from lengthy legal documents for more efficient processing.
- Medical Diagnosis Assistance: Focus on key symptoms and patient history while filtering out irrelevant personal information.
- Educational Q&A Systems: Improve the relevance of answers by focusing on the core educational content of student questions.
- Research Query Processing: Enhance literature review processes by focusing on the most relevant aspects of research questions.
When adapting this recipe to your specific use-case, consider:
- Fine-tuning the filtering process for your specific domain or types of queries.
- Experimenting with different prompt formats for both the filtering and answering stages.
- Implementing a feedback loop to continuously improve the quality of the filtering process.
- Combining S2A with other techniques like Chain of Thought or Self-Consistency for even more robust reasoning capabilities.
By leveraging Mirascope's call decorator, response models, and dynamic configuration, you can easily implement and customize the System to Attention technique to enhance your LLM's ability to focus on relevant information and provide more accurate responses across a wide range of applications.