Annotations

When you are viewing a trace in the full view, the right sidebar contains a section for discussions and annotations.

Discussion

The discussion tab is a place for you to leave open-ended comments on the trace. Often this is the first step towards annotating the trace. When you aren't yet sure how to annotate or trace (or if you want to), open-ended discussions are a great way to start.

You can use this to ask questions, leave feedback, discuss the trace with your team, or even just leave a note to yourself.

Assigning Traces For Annotation

In the trace view, annotations are read only. You can annotate a trace by assigning to a user, which adds it to their annotation queue.

Once added to the queue, you can annotate the trace by clicking the label, (optionally) providing a reason, and clicking submit.

When annotating a trace, existing annotations are not shown to you. This is to prevent biasing your annotation.

Annotating In Code

You can also annotate traces in code. This is useful when you want to annotate a trace using a custom script or UI.

import os

from lilypad import Lilypad

client = Lilypad()

client.ee.projects.annotations.create(
    project_uuid=os.environ["LILYPAD_PROJECT_ID"],
    body=[
        {
            "span_uuid": "...",
            "project_uuid": os.environ["LILYPAD_PROJECT_ID"],
            "function_uuid": "...",
            "label": "pass",  
            "reasoning": "this is a pass because...",  
            "type": "manual",
            "data": {},
            "assigned_to": ["..."],
            "assignee_email": ["[email protected]"],
        }
    ],
)

import os

from lilypad import Lilypad

client = Lilypad()

client.ee.projects.annotations.create(
    project_uuid=os.environ["LILYPAD_PROJECT_ID"],
    body=[
        {
            "span_uuid": "...",
            "project_uuid": os.environ["LILYPAD_PROJECT_ID"],
            "function_uuid": "...",
            "label": "pass",  
            "reasoning": "this is a pass because...",  
            "type": "manual",
            "data": {},
            "assigned_to": ["..."],
            "assignee_email": ["[email protected]"],
        }
    ],
)

Passthrough Annotations

Setting mode="wrap" in the trace decorator converts the original return type of the decorated function into a Traced[R] that provides methods for operating on the corresponding trace directly.

For example, you might want to enable passthrough annotation so that your users can annotate traces directly:

import lilypad

lilypad.configure()

@lilypad.trace(name="Answer Question", versioning="automatic", mode="wrap")  
def answer_question(question: str) -> str | None:
    response = client.models.generate_content(
        model="gemini-2.0-flash-001",
        contents=f"Answer this question: {question}",
    )
    return response.text
    
trace: lilypad.Trace[str | None] = answer_question("What is the capital of France?")
print(trace.response)  # original response
# > The capital of France is Paris.

trace.annotate(label="PASS", reason="The answer was correct")  

import lilypad

lilypad.configure()

@lilypad.trace(name="Answer Question", versioning="automatic", mode="wrap")  
def answer_question(question: str) -> str | None:
    response = client.models.generate_content(
        model="gemini-2.0-flash-001",
        contents=f"Answer this question: {question}",
    )
    return response.text
    
trace: lilypad.Trace[str | None] = answer_question("What is the capital of France?")
print(trace.response)  # original response
# > The capital of France is Paris.

trace.annotate(label="PASS", reason="The answer was correct")