Skip to content

dspy.RLM

RLM (Recursive Language Model) is a DSPy module that lets LLMs programmatically explore large contexts through a sandboxed Python REPL. Instead of feeding huge contexts directly into the prompt, RLM treats context as external data that the LLM examines via code execution and recursive sub-LLM calls.

This implements the approach described in “Recursive Language Models” (Zhang, Kraska, Khattab, 2025).

Migrating from CodeAct or ProgramOfThought

dspy.CodeAct and dspy.ProgramOfThought were deprecated in DSPy 3.4 and removed in DSPy 3.5, including their dspy.predict imports. Use dspy.RLM for code-executing programs. It is a replacement, not a compatibility alias.

Keep your task signature and named inputs, and replace the module constructor:

import dspy

# After configuring dspy.settings.lm:
solve = dspy.RLM("question -> answer")
result = solve(question="What is 12 factorial divided by the sum of primes below 30?")
print(result.answer)

def add(a: int, b: int) -> int:
    """Add two integers."""
    return a + b

# For programs that previously supplied tools:
agent = dspy.RLM("question -> answer", tools=[add])
  • RLM runs an iterative Python REPL and submits the signature’s output fields with SUBMIT(...). It does not use the old generated_code / finished protocol or the three code-generation, regeneration, and answer predictors. Custom prompts and code that accessed those internal predictors must be rewritten.
  • max_iters now limits REPL iterations, not just code-repair attempts. Review your iteration budget rather than assuming the old value has identical semantics. max_llm_calls separately limits sub-LM calls.
  • Pass tools with tools=[...]. RLM exposes them as host-side callables through the interpreter, rather than copying their source definitions into the sandbox. Host tools retain their own dependencies and privileges; sandbox restrictions do not apply to their host-side execution.
  • interpreter_factory=... and dspy.configure(interpreter_factory=...) remain supported. RLM requires an interpreter that supports its mutable tools dictionary and sandbox DSPy facade; see interpreter factories. RLM does not accept a caller-owned interpreter as a positional override.
  • Rebuild and re-optimize programs that used the removed modules. Their saved predictor states and internal trajectory formats are not interchangeable with RLM’s.

See the RLM deep dive for execution, outputs, and interpreter lifecycle details, and the installation instructions below for the default sandbox runtime.

When to Use RLM

As contexts grow, LLM performance degrades — a phenomenon known as context rot. RLMs address this by separating the variable space (information stored in the REPL) from the token space (what the LLM actually processes). The LLM dynamically loads only the context it needs, when it needs it.

Use RLM when:

  • Your context is too large to fit in the LLM’s context window effectively
  • The task benefits from programmatic exploration (searching, filtering, aggregating, chunking)
  • You need the LLM to decide how to decompose the problem, not you

Basic Usage

import dspy

dspy.configure(lm=dspy.LM("openai/gpt-5"))

# Create an RLM module
rlm = dspy.RLM("context, query -> answer")

# Call it like any other module
result = rlm(
    context="...very long document or data...",
    query="What is the total revenue mentioned?"
)
print(result.answer)

Deno Installation

RLM relies on Deno and Pyodide to create a local WASM sandbox for secure Python execution.

Install DSPy’s managed Deno runtime with pip install "dspy[deno]". See the PythonInterpreter installation guide for supported platforms, system-Deno fallback instructions, and dependency-isolation details.

Then you can run dspy.RLM.

You can also work with an external sandbox provider. We are still working on creating an example of using external sandbox providers.

How It Works

RLM operates in an iterative REPL loop:

  1. The LLM receives metadata about the context (type, length, preview) but not the full context
  2. The LLM writes Python code to explore the data (print samples, search, filter)
  3. Code executes in a sandboxed interpreter, and the LLM sees the output
  4. The LLM can call llm_query(prompt) to run sub-LLM calls for semantic analysis on snippets
  5. When done, the LLM calls SUBMIT(output) to return the final answer

What the LLM sees (step-by-step trace):

Step 1: Initial Metadata (no direct access to full context)

# Step 1: Peek at the data
print(context[:2000])
Output shown to the LLM:
[Preview of the first 2000 characters of the document]

Step 2: Write Code to Explore Context

# Step 2: Search for relevant sections
import re
matches = re.findall(r'revenue.*?\$[\d,]+', context, re.IGNORECASE)
print(matches)
Output shown to the LLM:
['Revenue in Q4: $5,000,000', 'Total revenue: $20,000,000']

Step 3: Trigger Sub-LLM Calls

# Step 3: Use sub-LLM for semantic extraction
result = llm_query(f"Extract the total revenue from: {matches[1]}")
print(result)
Output shown to the LLM:
$20,000,000

Step 4: Submit Final Answer

# Step 4: Return final answer
SUBMIT(result)
Output shown to the user:
$20,000,000

Constructor Parameters

Parameter Type Default Description
signature str \| Signature required Defines inputs and outputs (e.g., "context, query -> answer")
max_iters int 20 Maximum REPL interaction loops before fallback extraction
max_llm_calls int 50 Maximum llm_query/llm_query_batched calls per execution
max_output_chars int 10_000 Maximum characters to include from REPL output
verbose bool False Log detailed execution info
tools list[Union[Callable, dspy.Tool]] None Additional tool functions callable from interpreter code
sub_lm dspy.LM None LM for sub-queries. Defaults to dspy.settings.lm. Use a cheaper model here.
interpreter_factory Callable[[], CodeInterpreter] PythonInterpreter Creates one interpreter per invocation. RLM shuts down each returned interpreter. dspy.configure(interpreter_factory=...) replaces the default. A factory passed to the constructor wins, unless it is PythonInterpreter. May expose an optional execution_instructions string for the action prompt.

Built-in Tools

Inside the REPL, the LLM has access to:

Tool Description
llm_query(prompt) Query a sub-LLM for semantic analysis (~500K char capacity). prompt may be a list of strings and DSPy values, such as images
llm_query_batched(prompts) Query multiple prompts concurrently (faster for batch operations)
print() Print output (required to see results)
SUBMIT(...) Submit final output and end execution
Standard library re, json, collections, math, etc.

Examples

Long Document Q&A

import dspy

dspy.configure(lm=dspy.LM("openai/gpt-5"))

rlm = dspy.RLM("document, question -> answer", max_iters=10)

with open("large_report.txt") as f:
    document = f.read()  # 500K+ characters

result = rlm(
    document=document,
    question="What were the key findings from Q3?"
)
print(result.answer)

Using a Cheaper Sub-LM

import dspy

main_lm = dspy.LM("openai/gpt-5")
cheap_lm = dspy.LM("openai/gpt-5-nano")

dspy.configure(lm=main_lm)

# Root LM (gpt-5) decides strategy; sub-LM (gpt-5-nano) handles extraction
rlm = dspy.RLM("data, query -> summary", sub_lm=cheap_lm)

Multiple Typed Outputs

rlm = dspy.RLM("logs -> error_count: int, critical_errors: list[str]")

result = rlm(logs=server_logs)
print(f"Found {result.error_count} errors")
print(f"Critical: {result.critical_errors}")

Custom Tools

def fetch_metadata(doc_id: str) -> str:
    """Fetch metadata for a document ID."""
    return database.get_metadata(doc_id)

rlm = dspy.RLM(
    "documents, query -> answer",
    tools=[fetch_metadata]
)

Configuring the Interpreter

Pass the interpreter class directly when it needs no arguments. For configured construction, use any zero-argument callable, such as functools.partial:

from functools import partial

rlm = dspy.RLM(
    "context, query -> answer",
    interpreter_factory=partial(
        dspy.PythonInterpreter,
        enable_network_access=["example.com"],
    ),
)

One call puts the same interpreter behind every code-executing module in the program. Each invocation reads the setting, so dspy.context(interpreter_factory=...) scopes the choice and dspy.configure(...) reaches modules built before the call. A factory passed to a constructor still wins, unless it is PythonInterpreter:

dspy.configure(interpreter_factory=MyInterpreter)

RLM creates and shuts down one interpreter from this factory per invocation. It adds invocation-scoped tools to the returned interpreter’s mutable tools dictionary, so remote sandboxes need a CodeInterpreter adapter that supports that protocol.

To override the factory for a single invocation, pass a zero-argument callable via the interpreter_factory keyword:

result = rlm(context=data, query=query, interpreter_factory=MyInterpreter)
result = await rlm.acall(context=data, query=query, interpreter_factory=MyInterpreter)

The call-time factory takes precedence over constructor and global/context factories, including when explicitly passing PythonInterpreter. RLM calls it once and shuts down the returned interpreter on success or failure. The factory must return a fresh interpreter each time. Passing a live interpreter at call time is no longer supported; migrate rlm(interpreter, ...) to rlm(..., interpreter_factory=factory). The option is keyword-only, and interpreter_factory is reserved for runtime configuration rather than signature inputs.

If the factory exposes an execution_instructions string, RLM adds it to the action predictor’s task instructions, which DSPy adapters place in the system prompt. Optimizers such as GEPA may therefore adapt the execution guidance along with the rest of the action policy. Factory classes and configured callable provider objects can expose this metadata; anonymous factories without it continue to use the generic action prompt. RLM refreshes these instructions for each action call, so a dspy.context(interpreter_factory=...) override keeps the prompt aligned with the runtime that executes the generated code.

Custom Sandbox-Serializable Inputs

For inputs that should be loaded into the sandbox differently from normal Python values, subclass dspy.SandboxSerializable. RLM detects these inputs, sends their serialized payload into the interpreter, runs their setup code, and exposes the reconstructed value under the original input name.

The value crosses on the same path as every other value the interpreter’s JSON can’t carry (dspy.Image and other dspy types): it is rebuilt once per call, so changes the code makes to it persist across turns. The same holds for a dspy.Flex input, and for a provided tool’s result. In RLM, a tool’s result is rebuilt only when the tool’s return annotation names its type (for example -> DataFrame); otherwise RLM reports an error rather than passing the value’s string. Three limits:

  • Only host values are rebuilt. A SandboxSerializable defined by sandboxed code and passed to a host tool is refused: the host never runs code the sandbox wrote.
  • It is one-way. A value that comes back from the sandbox (via SUBMIT or a tool argument) is whatever the code returns, not a rebuilt SandboxSerializable.
  • It needs dspy inside the interpreter. An interpreter that can’t load it (for example a restricted compiler, or a sandbox without its cached bundle offline) raises an error for these inputs.
class DataFrame(dspy.SandboxSerializable):
    def sandbox_setup(self) -> str:
        return "import pandas as pd\nimport base64\nimport io"

    def to_sandbox(self) -> bytes:
        return base64.b64encode(self.data.to_parquet(index=False))

    def sandbox_assignment(self, var_name: str, data_expr: str) -> str:
        return f"{var_name} = pd.read_parquet(io.BytesIO(base64.b64decode({data_expr})))"

    def rlm_preview(self, max_chars: int = 500) -> str:
        return f"DataFrame: {self.data.shape[0]} rows x {self.data.shape[1]} columns"

SandboxSerializable also defines a Pydantic schema hook so subclasses can be used directly in DSPy signatures, for example data: DataFrame = dspy.InputField(). The hook is intentionally pass-through: Pydantic accepts the object as-is and serializes it with str(value) for schema/metadata purposes. RLM’s real sandbox transport still comes from to_sandbox() and sandbox_assignment().

Images and Other DSPy Types

Values of DSPy types such as dspy.Image keep their type across the REPL boundary: in inputs, tool arguments and results, SUBMIT, llm_query prompt lists, and sub-agents built with dspy in the REPL. The REPL runs dspy itself, so a dspy.Image is a dspy.Image there, with its methods. Your own dspy.Type subclasses and models arrive as plain data of their fields, with their methods left on the host, as in dspy.Flex. A tool’s result keeps its type when the tool’s return annotation names it (-> dspy.Image).

dspy.Image.to_pil() needs Pillow in the interpreter. dspy.LocalInterpreter uses your environment’s. PythonInterpreter’s sandbox includes Pyodide’s build of Pillow, whether or not your environment has Pillow. Its sandbox_packages choose the packages Pyodide ships that the REPL can import, by default ("pillow",):

from functools import partial

import dspy

# numpy as well as Pillow
rlm = dspy.RLM(
    "photo: dspy.Image, question -> answer",
    interpreter_factory=partial(
        dspy.PythonInterpreter,
        sandbox_packages=["pillow", "numpy"],
    ),
)
# no packages, for a sandbox that starts slightly faster;
# images still pass through, but to_pil() raises ImportError
dspy.configure(
    interpreter_factory=partial(
        dspy.PythonInterpreter,
        sandbox_packages=[],
    ),
)
def take_screenshot() -> dspy.Image:
    ...

rlm = dspy.RLM("photo: dspy.Image, question -> answer, crop: dspy.Image", tools=[take_screenshot])

The model can then write code such as:

crop = dspy.Image(photo.to_pil().crop((600, 400, 900, 600)))
label = llm_query(["Read the label in this crop.", crop])
screen = take_screenshot()  # a dspy.Image
answer = dspy.Predict("image: dspy.Image, question -> answer")(image=screen, question=question).answer
SUBMIT(answer, crop)

Async Execution

import asyncio

rlm = dspy.RLM("context, query -> answer")

async def process():
    result = await rlm.acall(context=data, query="Summarize this")
    return result.answer

answer = asyncio.run(process())

Inspecting the Trajectory

result = rlm(context=data, query="Find the magic number")

# See what code the LLM executed
for step in result.trajectory:
    print(f"Code:\n{step['code']}")
    print(f"Output:\n{step['output']}\n")

Output

RLM returns a Prediction with:

  • Output fields from your signature (e.g., result.answer)
  • trajectory: List of dicts with reasoning, code, output for each step
  • final_reasoning: The LLM’s reasoning on the final step

Notes

Thread Safety

Constructor and call-time factories may be called concurrently and must return a fresh interpreter each time. RLM retains that interpreter throughout one invocation and shuts it down afterward. PythonInterpreter must stay on the thread where it was first used.

Interpreter Requirements

RLM defaults to PythonInterpreter, which requires Deno to be installed for the Pyodide WASM sandbox. LocalInterpreter requires no additional runtime and provides a separate CPython process, but it is not a security sandbox: generated code retains the host user’s files, environment, credentials, network, and process authority.

API Reference

dspy.RLM(signature: type[Signature] | str, max_iters: int = 20, max_llm_calls: int = 50, max_output_chars: int = 10000, verbose: bool = False, tools: list[Callable] | None = None, sub_lm: dspy.LM | None = None, interpreter_factory: Callable[[], CodeInterpreter] = PythonInterpreter, *, history_processor: Callable[[dspy.History], dspy.History | None] | None = None)

Bases: Module

Recursive Language Model module.

Uses a sandboxed REPL to let the LLM programmatically explore large contexts through code execution. The LLM writes Python code to examine data, call sub-LLMs for semantic analysis, and build up answers iteratively.

interpreter_factory defaults to PythonInterpreter (Deno/Pyodide/WASM), and dspy.configure(interpreter_factory=...) replaces that default. Either route accepts an adapter for a remote sandbox. RLM updates the interpreter’s mutable tools dictionary with invocation-scoped tools before execution. Pass a zero-argument factory via interpreter_factory= at call time to override the runtime for one invocation. RLM shuts down every interpreter it creates.

Examples:

# Basic usage
rlm = dspy.RLM("context, query -> output", max_iters=10)
result = rlm(context="...very long text...", query="What is the magic number?")
print(result.output)

Parameters:

Name Type Description Default
signature type[Signature] | str

Defines inputs and outputs. String like “context, query -> answer” or a Signature class.

required
max_iters int

Maximum REPL interaction iterations.

20
max_llm_calls int

Maximum sub-LLM calls (llm_query/llm_query_batched and sub-agent LM calls) per execution.

50
max_output_chars int

Maximum characters to include from REPL output.

10000
verbose bool

Whether to log detailed execution info.

False
history_processor Callable[[History], History | None] | None

Optional callable applied to the conversation history before each iteration (e.g. to truncate or summarize it). It receives a deep copy of the current dspy.History and may either return a new history or edit the copy in place and return None. If it raises, the error is logged and the iteration proceeds with the unprocessed history.

None
tools list[Callable] | None

List of tool functions or dspy.Tool objects callable from interpreter code. Built-in tools: llm_query(prompt), llm_query_batched(prompts).

None
sub_lm LM | None

LM for llm_query/llm_query_batched and sub-agents. Defaults to dspy.settings.lm. Allows using a different (e.g., cheaper) model for sub-queries.

None
interpreter_factory Callable[[], CodeInterpreter]

Zero-argument callable that creates an interpreter for each forward pass. The callable may be invoked concurrently, and DSPy shuts down each interpreter it returns. RLM updates the returned interpreter’s mutable tools dictionary before execution. The callable may expose an execution_instructions string describing its runtime for the action prompt. RLM installs the sandbox dspy facade into every interpreter it creates, so the runtime must be able to host it (see CodeInterpreter). RLM applies the active factory’s instructions to each action call, so dspy.context can switch runtimes without changing the shared predictor signature. Defaults to dspy.PythonInterpreter; dspy.configure(interpreter_factory=...) replaces the default.

PythonInterpreter
Source code in dspy/predict/rlm.py
def __init__(
    self,
    signature: type[Signature] | str,
    max_iters: int = 20,
    max_llm_calls: int = 50,
    max_output_chars: int = 10_000,
    verbose: bool = False,
    tools: list[Callable] | None = None,
    sub_lm: dspy.LM | None = None,
    interpreter_factory: Callable[[], CodeInterpreter] = PythonInterpreter,
    *,
    history_processor: Callable[[dspy.History], dspy.History | None] | None = None,
):
    """
    Args:
        signature: Defines inputs and outputs. String like "context, query -> answer"
                  or a Signature class.
        max_iters: Maximum REPL interaction iterations.
        max_llm_calls: Maximum sub-LLM calls (llm_query/llm_query_batched and sub-agent LM calls) per execution.
        max_output_chars: Maximum characters to include from REPL output.
        verbose: Whether to log detailed execution info.
        history_processor: Optional callable applied to the conversation history before each iteration
            (e.g. to truncate or summarize it). It receives a deep copy of the current `dspy.History`
            and may either return a new history or edit the copy in place and return `None`.
            If it raises, the error is logged and the iteration proceeds with the unprocessed history.
        tools: List of tool functions or dspy.Tool objects callable from interpreter code.
              Built-in tools: llm_query(prompt), llm_query_batched(prompts).
        sub_lm: LM for llm_query/llm_query_batched and sub-agents. Defaults to dspy.settings.lm.
               Allows using a different (e.g., cheaper) model for sub-queries.
        interpreter_factory: Zero-argument callable that creates an interpreter for each forward pass. The
            callable may be invoked concurrently, and DSPy shuts down each interpreter it returns. RLM updates
            the returned interpreter's mutable ``tools`` dictionary before execution. The callable may expose
            an ``execution_instructions`` string describing its runtime for the action prompt. RLM installs the
            sandbox dspy facade into every interpreter it creates, so the runtime must be able to host it
            (see ``CodeInterpreter``). RLM applies the
            active factory's instructions to each action call, so ``dspy.context`` can switch runtimes
            without changing the shared predictor signature. Defaults to ``dspy.PythonInterpreter``;
            ``dspy.configure(interpreter_factory=...)`` replaces the default.
    """
    super().__init__()
    _validate_interpreter_factory(interpreter_factory)
    self.history_processor = history_processor
    self.signature = ensure_signature(signature)
    if any(
        kind.extract_custom_type_from_annotation(field.rebuild_annotation())
        for field in self.signature.output_fields.values()
        for kind in (Noul, Choice, Score)
    ):
        warnings.warn(
            "RLM support for Noul, Choice, and Score outputs is not implemented consistently: "
            "decision evidence decoding is not guaranteed, including for nested output types. "
            "Use Predict with top-level decision outputs instead.",
            UserWarning,
            stacklevel=2,
        )
    self.max_iters = max_iters
    self.max_llm_calls = max_llm_calls
    self.max_output_chars = max_output_chars
    self.verbose = verbose
    self.sub_lm = sub_lm
    self._interpreter_factory = interpreter_factory
    self._user_tools = self._normalize_tools(tools)
    self._validate_namespace(self._user_tools)

    # Build the action and extract signatures
    action_sig, extract_sig = self._build_signatures()
    self._action_signature = action_sig
    self.generate_action = dspy.Predict(action_sig)
    self.extract = dspy.Predict(extract_sig)

Attributes

tools: dict[str, Tool] property

User-provided tools (excludes internal llm_query/llm_query_batched).

Methods:

__call__(*args, **kwargs) -> Prediction
Source code in dspy/primitives/module.py
@with_callbacks
def __call__(self, *args, **kwargs) -> Prediction:
    from dspy.dsp.utils.settings import thread_local_overrides

    caller_modules = settings.caller_modules or []
    caller_modules = list(caller_modules)
    caller_modules.append(self)

    with settings.context(caller_modules=caller_modules):
        if settings.track_usage and thread_local_overrides.get().get("usage_tracker") is None:
            with track_usage() as usage_tracker:
                output = self.forward(*args, **kwargs)
            tokens = usage_tracker.get_total_tokens()
            self._set_lm_usage(tokens, output)

            return output

        return self.forward(*args, **kwargs)
forward(*, interpreter_factory: Callable[[], CodeInterpreter] | None = None, history_processor: Callable[[dspy.History], dspy.History | None] | None = None, **input_args) -> Prediction

Execute RLM to produce outputs from the given inputs.

Parameters:

Name Type Description Default
interpreter_factory Callable[[], CodeInterpreter] | None

Optional zero-argument factory, passed by keyword. Overrides the constructor and configured factories for this invocation. Must return a fresh interpreter; RLM injects tools and output metadata and shuts it down on exit, including failures. Sub-agents built in the sandbox run their own code on this factory too.

None
history_processor Callable[[History], History | None] | None

Optional callable, passed by keyword. Overrides the constructor’s processor for this invocation; see the class docstring for the contract.

None
**input_args

Input values matching the signature’s input fields. An optional history (dspy.History) is treated as read-only input; it is never mutated.

{}

Returns:

Type Description
Prediction

Prediction with the signature’s outputs, history, final_reasoning, and this call’s trajectory.

Prediction

history is a new dspy.History containing the input history plus one event per REPL

Prediction

iteration; pass it back as history= to continue the conversation. History does not restore

Prediction

interpreter variables or mutations; each invocation uses a fresh interpreter.

Raises:

Type Description
ValueError

If required input fields are missing

CodeInterpreterError

If interpreter setup, process, or protocol fails

Source code in dspy/predict/rlm.py
def forward(
    self,
    *,
    interpreter_factory: Callable[[], CodeInterpreter] | None = None,
    history_processor: Callable[[dspy.History], dspy.History | None] | None = None,
    **input_args,
) -> Prediction:
    """Execute RLM to produce outputs from the given inputs.

    Args:
        interpreter_factory: Optional zero-argument factory, passed by keyword. Overrides the constructor
            and configured factories for this invocation. Must return a fresh interpreter; RLM injects tools
            and output metadata and shuts it down on exit, including failures. Sub-agents built in the
            sandbox run their own code on this factory too.
        history_processor: Optional callable, passed by keyword. Overrides the constructor's processor for
            this invocation; see the class docstring for the contract.
        **input_args: Input values matching the signature's input fields. An optional `history`
            (`dspy.History`) is treated as read-only input; it is never mutated.

    Returns:
        Prediction with the signature's outputs, `history`, `final_reasoning`, and this call's `trajectory`.
        `history` is a new `dspy.History` containing the input history plus one event per REPL
        iteration; pass it back as `history=` to continue the conversation. History does not restore
        interpreter variables or mutations; each invocation uses a fresh interpreter.

    Raises:
        ValueError: If required input fields are missing
        CodeInterpreterError: If interpreter setup, process, or protocol fails
    """
    self._validate_inputs(input_args)
    if interpreter_factory is None:
        interpreter_factory = resolve_interpreter_factory(self._interpreter_factory)

    history_processor = history_processor if history_processor else self.history_processor

    output_field_names = list(self.signature.output_fields.keys())
    budget = _LLMCallBudget(self.max_llm_calls)
    # The caller's history is input only; RLM appends iteration events to a private copy that is returned
    # as `result.history`.
    history = _coerce_history(input_args.pop("history", None)).model_copy(deep=True)
    typed = self._crosses_wire_types(input_args)
    execution_tools = self._prepare_execution_tools(budget, typed)

    variables = self._build_variables(**input_args)

    with (
        self._interpreter_context(execution_tools, interpreter_factory) as repl,
        self._setup_facade(repl, budget, interpreter_factory, typed) as sub_agent_rules,
    ):
        regular_args = self._prepare_serializable_vars(input_args, repl)
        turn_entries: list[REPLEntry] = []
        variables_info = [variable.format() for variable in variables]

        inputs = {name: input_args[name] for name in self.signature.input_fields if name in input_args}

        for iteration in range(self.max_iters):
            history = _apply_history_processor(history_processor, history)

            repl_entry, final_outputs = self._execute_iteration(
                repl,
                variables,
                history,
                iteration,
                regular_args,
                output_field_names,
                interpreter_factory,
                sub_agent_rules,
            )
            turn_entries.append(repl_entry)

            # Update history
            history_event = self._repl_entry_event(
                inputs=inputs, variables_info=variables_info, iteration=iteration, repl_entry=repl_entry
            )
            self._append_history_with_event(history, history_event)

            if final_outputs is not None:
                history.messages[-1].update(final_outputs)  # Add output fields to last history event
                return Prediction(
                    **final_outputs,
                    history=history,
                    final_reasoning=repl_entry.reasoning,
                    trajectory=[e.to_trajectory_dict() for e in turn_entries],
                )

        # Max iterations reached - use extract fallback
        return self._extract_fallback(
            variables,
            history,
            turn_entries,
            output_field_names,
        )
aforward(*, interpreter_factory: Callable[[], CodeInterpreter] | None = None, history_processor: Callable[[dspy.History], dspy.History | None] | None = None, **input_args) -> Prediction async

Async version of forward(). Execute RLM to produce outputs.

Parameters:

Name Type Description Default
interpreter_factory Callable[[], CodeInterpreter] | None

Optional zero-argument factory, passed by keyword. Overrides the constructor and configured factories for this invocation. Must return a fresh interpreter; RLM injects tools and output metadata and shuts it down on exit, including failures. Sub-agents built in the sandbox run their own code on this factory too.

None
history_processor Callable[[History], History | None] | None

Optional callable, passed by keyword. Overrides the constructor’s processor for this invocation; see the class docstring for the contract.

None
**input_args

Input values matching the signature’s input fields. An optional history (dspy.History) is treated as read-only input; it is never mutated.

{}

Returns:

Type Description
Prediction

Prediction with the signature’s outputs, history, final_reasoning, and this call’s trajectory.

Prediction

history is a new dspy.History containing the input history plus one event per REPL

Prediction

iteration; pass it back as history= to continue the conversation. History does not restore

Prediction

interpreter variables or mutations; each invocation uses a fresh interpreter.

Raises:

Type Description
ValueError

If required input fields are missing

CodeInterpreterError

If interpreter setup, process, or protocol fails

Source code in dspy/predict/rlm.py
async def aforward(
    self,
    *,
    interpreter_factory: Callable[[], CodeInterpreter] | None = None,
    history_processor: Callable[[dspy.History], dspy.History | None] | None = None,
    **input_args,
) -> Prediction:
    """Async version of forward(). Execute RLM to produce outputs.

    Args:
        interpreter_factory: Optional zero-argument factory, passed by keyword. Overrides the constructor
            and configured factories for this invocation. Must return a fresh interpreter; RLM injects tools
            and output metadata and shuts it down on exit, including failures. Sub-agents built in the
            sandbox run their own code on this factory too.
        history_processor: Optional callable, passed by keyword. Overrides the constructor's processor for
            this invocation; see the class docstring for the contract.
        **input_args: Input values matching the signature's input fields. An optional `history`
            (`dspy.History`) is treated as read-only input; it is never mutated.

    Returns:
        Prediction with the signature's outputs, `history`, `final_reasoning`, and this call's `trajectory`.
        `history` is a new `dspy.History` containing the input history plus one event per REPL
        iteration; pass it back as `history=` to continue the conversation. History does not restore
        interpreter variables or mutations; each invocation uses a fresh interpreter.

    Raises:
        ValueError: If required input fields are missing
        CodeInterpreterError: If interpreter setup, process, or protocol fails
    """
    self._validate_inputs(input_args)
    if interpreter_factory is None:
        interpreter_factory = resolve_interpreter_factory(self._interpreter_factory)

    history_processor = history_processor if history_processor else self.history_processor

    output_field_names = list(self.signature.output_fields.keys())
    budget = _LLMCallBudget(self.max_llm_calls)
    # The caller's history is input only; RLM appends iteration events to a private copy that is returned
    # as `result.history`.
    history = _coerce_history(input_args.pop("history", None)).model_copy(deep=True)
    typed = self._crosses_wire_types(input_args)
    execution_tools = self._prepare_execution_tools(budget, typed)

    variables = self._build_variables(**input_args)

    with (
        self._interpreter_context(execution_tools, interpreter_factory) as repl,
        self._setup_facade(repl, budget, interpreter_factory, typed) as sub_agent_rules,
    ):
        regular_args = self._prepare_serializable_vars(input_args, repl)
        turn_entries: list[REPLEntry] = []
        variables_info = [variable.format() for variable in variables]

        inputs = {name: input_args[name] for name in self.signature.input_fields if name in input_args}

        for iteration in range(self.max_iters):
            history = _apply_history_processor(history_processor, history)
            repl_entry, final_outputs = await self._aexecute_iteration(
                repl,
                variables,
                history,
                iteration,
                regular_args,
                output_field_names,
                interpreter_factory,
                sub_agent_rules,
            )
            turn_entries.append(repl_entry)

            # Update history
            history_event = self._repl_entry_event(
                inputs=inputs, variables_info=variables_info, iteration=iteration, repl_entry=repl_entry
            )
            self._append_history_with_event(history, history_event)

            if final_outputs is not None:
                history.messages[-1].update(final_outputs)  # Add output fields to last history event
                return Prediction(
                    **final_outputs,
                    history=history,
                    final_reasoning=repl_entry.reasoning,
                    trajectory=[e.to_trajectory_dict() for e in turn_entries],
                )

        # Max iterations reached - use extract fallback
        return await self._aextract_fallback(variables, history, turn_entries, output_field_names)
batch(examples: list[Example], num_threads: int | None = None, max_errors: int | None = None, return_failed_examples: bool = False, provide_traceback: bool | None = None, disable_progress_bar: bool = False, timeout: int = 120, straggler_limit: int = 3) -> list[Example] | tuple[list[Example], list[Example], list[Exception]]

Processes a list of dspy.Example instances in parallel using the Parallel module.

Parameters:

Name Type Description Default
examples list[Example]

List of dspy.Example instances to process.

required
num_threads int | None

Number of threads to use for parallel processing.

None
max_errors int | None

Maximum number of errors allowed before stopping execution. If None, inherits from dspy.settings.max_errors.

None
return_failed_examples bool

Whether to return failed examples and exceptions.

False
provide_traceback bool | None

Whether to include traceback information in error logs.

None
disable_progress_bar bool

Whether to display the progress bar.

False
timeout int

Seconds before a straggler task is resubmitted. Set to 0 to disable.

120
straggler_limit int

Only check for stragglers when this many or fewer tasks remain.

3

Returns:

Type Description
list[Example] | tuple[list[Example], list[Example], list[Exception]]

List of results, and optionally failed examples and exceptions.

Source code in dspy/primitives/module.py
def batch(
    self,
    examples: list[Example],
    num_threads: int | None = None,
    max_errors: int | None = None,
    return_failed_examples: bool = False,
    provide_traceback: bool | None = None,
    disable_progress_bar: bool = False,
    timeout: int = 120,
    straggler_limit: int = 3,
) -> list[Example] | tuple[list[Example], list[Example], list[Exception]]:
    """
    Processes a list of dspy.Example instances in parallel using the Parallel module.

    Args:
        examples: List of dspy.Example instances to process.
        num_threads: Number of threads to use for parallel processing.
        max_errors: Maximum number of errors allowed before stopping execution.
            If ``None``, inherits from ``dspy.settings.max_errors``.
        return_failed_examples: Whether to return failed examples and exceptions.
        provide_traceback: Whether to include traceback information in error logs.
        disable_progress_bar: Whether to display the progress bar.
        timeout: Seconds before a straggler task is resubmitted. Set to 0 to disable.
        straggler_limit: Only check for stragglers when this many or fewer tasks remain.

    Returns:
        List of results, and optionally failed examples and exceptions.
    """
    # Create a list of execution pairs (self, example)
    exec_pairs = [(self, example.inputs()) for example in examples]

    # Create an instance of Parallel
    parallel_executor = Parallel(
        num_threads=num_threads,
        max_errors=max_errors,
        return_failed_examples=return_failed_examples,
        provide_traceback=provide_traceback,
        disable_progress_bar=disable_progress_bar,
        timeout=timeout,
        straggler_limit=straggler_limit,
    )

    # Execute the forward method of Parallel
    if return_failed_examples:
        results, failed_examples, exceptions = parallel_executor.forward(exec_pairs)
        return results, failed_examples, exceptions
    else:
        results = parallel_executor.forward(exec_pairs)
        return results
deepcopy()

Deep copy the module.

This is a tweak to the default python deepcopy that only deep copies self.parameters(), and for other attributes, we just do the shallow copy.

Source code in dspy/primitives/base_module.py
def deepcopy(self):
    """Deep copy the module.

    This is a tweak to the default python deepcopy that only deep copies `self.parameters()`, and for other
    attributes, we just do the shallow copy.
    """
    try:
        # If the instance itself is copyable, we can just deep copy it.
        # Otherwise we will have to create a new instance and copy over the attributes one by one.
        return copy.deepcopy(self)
    except Exception:
        pass

    # Create an empty instance.
    new_instance = self.__class__.__new__(self.__class__)
    # Set attribuetes of the copied instance.
    for attr, value in self.__dict__.items():
        if isinstance(value, BaseModule):
            setattr(new_instance, attr, value.deepcopy())
        else:
            try:
                # Try to deep copy the attribute
                setattr(new_instance, attr, copy.deepcopy(value))
            except Exception:
                logging.warning(
                    f"Failed to deep copy attribute '{attr}' of {self.__class__.__name__}, "
                    "falling back to shallow copy or reference copy."
                )
                try:
                    # Fallback to shallow copy if deep copy fails
                    setattr(new_instance, attr, copy.copy(value))
                except Exception:
                    # If even the shallow copy fails, we just copy over the reference.
                    setattr(new_instance, attr, value)

    return new_instance
dump_state(json_mode=True)
Source code in dspy/primitives/base_module.py
def dump_state(self, json_mode=True):
    return {name: param.dump_state(json_mode=json_mode) for name, param in self.named_parameters()}
get_lm()

Get the language model used by this module’s predictors.

Returns the language model if every module leaf uses the same LM. Raises an error if multiple different LMs are in use.

Returns:

Type Description

The language model instance used by this module’s predictors.

Raises:

Type Description
ValueError

If multiple different language models are being used by the predictors in this module.

Source code in dspy/primitives/module.py
def get_lm(self):
    """Get the language model used by this module's predictors.

    Returns the language model if every module leaf uses the same LM. Raises an error if multiple
    different LMs are in use.

    Returns:
        The language model instance used by this module's predictors.

    Raises:
        ValueError: If multiple different language models are being
            used by the predictors in this module.
    """
    all_used_lms = [param.lm for _, param in self.named_parameters() if isinstance(param, Module)]

    if len(set(all_used_lms)) == 1:
        return all_used_lms[0]

    raise ValueError("Multiple LMs are being used in the module. There's no unique LM to return.")
load(path, allow_pickle=False, allow_unsafe_lm_state=False)

Load the saved module. You may also want to check out dspy.load, if you want to load an entire program, not just the state for an existing program.

Parameters:

Name Type Description Default
path str

Path to the saved state file, which should be a .json or a .pkl file

required
allow_pickle bool

If True, allow loading .pkl files, which can run arbitrary code. This is dangerous and should only be used if you are sure about the source of the file and in a trusted environment.

False
allow_unsafe_lm_state bool

If True, preserves unsafe LM endpoint keys (e.g., api_base, base_url, and model_list) from loaded state and allows importing custom LM classes. Enable only for trusted files.

False
Source code in dspy/primitives/base_module.py
def load(self, path, allow_pickle=False, allow_unsafe_lm_state=False):
    """Load the saved module. You may also want to check out dspy.load, if you want to
    load an entire program, not just the state for an existing program.

    Args:
        path (str): Path to the saved state file, which should be a .json or a .pkl file
        allow_pickle (bool): If True, allow loading .pkl files, which can run arbitrary code.
            This is dangerous and should only be used if you are sure about the source of the file and in a trusted environment.
        allow_unsafe_lm_state (bool): If True, preserves unsafe LM endpoint keys (e.g.,
            `api_base`, `base_url`, and `model_list`) from loaded state and allows importing custom LM classes.
            Enable only for trusted files.
    """
    path = Path(path)

    if path.suffix == ".json":
        with open(path, "rb") as f:
            state = orjson.loads(f.read())
    elif path.suffix == ".pkl":
        if not allow_pickle:
            raise ValueError("Loading .pkl files can run arbitrary code, which may be dangerous. Prefer "
                             "saving with .json files if possible. Set `allow_pickle=True` "
                             "if you are sure about the source of the file and in a trusted environment.")
        with open(path, "rb") as f:
            state = cloudpickle.load(f)
    else:
        raise ValueError(f"`path` must end with `.json` or `.pkl`, but received: {path}")

    dependency_versions = get_dependency_versions()
    saved_dependency_versions = state["metadata"]["dependency_versions"]
    for key, saved_version in saved_dependency_versions.items():
        if dependency_versions[key] != saved_version:
            logger.warning(
                f"There is a mismatch of {key} version between saved model and current environment. "
                f"You saved with `{key}=={saved_version}`, but now you have "
                f"`{key}=={dependency_versions[key]}`. This might cause errors or performance downgrade "
                "on the loaded model, please consider loading the model in the same environment as the "
                "saving environment."
            )
    self.load_state(state, allow_unsafe_lm_state=allow_unsafe_lm_state)
load_state(state, *, allow_unsafe_lm_state=False)
Source code in dspy/primitives/base_module.py
def load_state(self, state, *, allow_unsafe_lm_state=False):
    from dspy import Module

    def _apply(module):
        for name, param in module.named_parameters():
            if isinstance(param, Module):
                param.load_state(state[name], allow_unsafe_lm_state=allow_unsafe_lm_state)
            else:
                param.load_state(state[name])

    _apply(self.deepcopy())  # trial run raises before self is touched
    _apply(self)
named_parameters()

Unlike PyTorch, handles (non-recursive) lists of parameters too.

Source code in dspy/primitives/base_module.py
def named_parameters(self):
    """
    Unlike PyTorch, handles (non-recursive) lists of parameters too.
    """

    import dspy
    from dspy.predict.parameter import Parameter

    visited = set()
    named_parameters = []

    def add_parameter(param_name, param_value):
        if isinstance(param_value, Parameter):
            if id(param_value) not in visited:
                visited.add(id(param_value))
                named_parameters.append((param_name, param_value))

        elif isinstance(param_value, dspy.Module):
            # When a sub-module is pre-compiled, keep it frozen.
            if not getattr(param_value, "_compiled", False):
                for sub_name, param in param_value.named_parameters():
                    add_parameter(f"{param_name}.{sub_name}", param)

    if isinstance(self, Parameter):
        add_parameter("self", self)

    for name, value in self.__dict__.items():
        if isinstance(value, Parameter):
            add_parameter(name, value)

        elif isinstance(value, dspy.Module):
            # When a sub-module is pre-compiled, keep it frozen.
            if not getattr(value, "_compiled", False):
                for sub_name, param in value.named_parameters():
                    add_parameter(f"{name}.{sub_name}", param)

        elif isinstance(value, (list, tuple)):
            for idx, item in enumerate(value):
                add_parameter(f"{name}[{idx}]", item)

        elif isinstance(value, dict):
            for key, item in value.items():
                add_parameter(f"{name}['{key}']", item)

    return named_parameters
named_predictors()

Return all named Predict modules in this module.

Iterates through all parameters and returns those that are instances of dspy.Predict, along with their names.

Returns:

Type Description

list[tuple[str, Predict]]: A list of (name, predictor) tuples where name is the attribute path and predictor is the Predict instance.

Examples:

>>> import dspy
>>> class MyProgram(dspy.Module):
...     def __init__(self):
...         super().__init__()
...         self.qa = dspy.Predict("question -> answer")
...         self.summarize = dspy.Predict("text -> summary")
...
>>> program = MyProgram()
>>> for name, p in program.named_predictors():
...     print(name)
qa
summarize
Source code in dspy/primitives/module.py
def named_predictors(self):
    """Return all named Predict modules in this module.

    Iterates through all parameters and returns those that are instances
    of ``dspy.Predict``, along with their names.

    Returns:
        list[tuple[str, Predict]]: A list of (name, predictor) tuples
            where name is the attribute path and predictor is the
            Predict instance.

    Examples:
        >>> import dspy
        >>> class MyProgram(dspy.Module):
        ...     def __init__(self):
        ...         super().__init__()
        ...         self.qa = dspy.Predict("question -> answer")
        ...         self.summarize = dspy.Predict("text -> summary")
        ...
        >>> program = MyProgram()
        >>> for name, p in program.named_predictors():
        ...     print(name)
        qa
        summarize
    """
    from dspy.predict.predict import Predict

    return [(name, param) for name, param in self.named_parameters() if isinstance(param, Predict)]
parameters()
Source code in dspy/primitives/base_module.py
def parameters(self):
    return [param for _, param in self.named_parameters()]
predictors()

Return all Predict modules in this module.

Returns:

Type Description

list[Predict]: A list of all Predict instances in this module.

Examples:

>>> import dspy
>>> class MyProgram(dspy.Module):
...     def __init__(self):
...         super().__init__()
...         self.qa = dspy.Predict("question -> answer")
...
>>> program = MyProgram()
>>> len(program.predictors())
1
Source code in dspy/primitives/module.py
def predictors(self):
    """Return all Predict modules in this module.

    Returns:
        list[Predict]: A list of all Predict instances in this module.

    Examples:
        >>> import dspy
        >>> class MyProgram(dspy.Module):
        ...     def __init__(self):
        ...         super().__init__()
        ...         self.qa = dspy.Predict("question -> answer")
        ...
        >>> program = MyProgram()
        >>> len(program.predictors())
        1
    """
    return [param for _, param in self.named_predictors()]
reset_copy()

Deep copy the module and reset all parameters.

Source code in dspy/primitives/base_module.py
def reset_copy(self):
    """Deep copy the module and reset all parameters."""
    new_instance = self.deepcopy()

    for param in new_instance.parameters():
        param.reset()

    return new_instance
save(path, save_program=False, modules_to_serialize=None)

Save the module.

Save the module to a directory or a file. There are two modes: - save_program=False: Save only the state of the module to a json or pickle file, based on the value of the file extension. - save_program=True: Save the whole module to a directory via cloudpickle, which contains both the state and architecture of the model.

If save_program=True and modules_to_serialize are provided, it will register those modules for serialization with cloudpickle’s register_pickle_by_value. This causes cloudpickle to serialize the module by value rather than by reference, ensuring the module is fully preserved along with the saved program. This is useful when you have custom modules that need to be serialized alongside your program. If None, then no modules will be registered for serialization.

We also save the dependency versions, so that the loaded model can check if there is a version mismatch on critical dependencies or DSPy version.

Parameters:

Name Type Description Default
path str

Path to the saved state file, which should be a .json or .pkl file when save_program=False, and a directory when save_program=True.

required
save_program bool

If True, save the whole module to a directory via cloudpickle, otherwise only save the state.

False
modules_to_serialize list

A list of modules to serialize with cloudpickle’s register_pickle_by_value. If None, then no modules will be registered for serialization.

None
Source code in dspy/primitives/base_module.py
def save(self, path, save_program=False, modules_to_serialize=None):
    """Save the module.

    Save the module to a directory or a file. There are two modes:
    - `save_program=False`: Save only the state of the module to a json or pickle file, based on the value of
        the file extension.
    - `save_program=True`: Save the whole module to a directory via cloudpickle, which contains both the state and
        architecture of the model.

    If `save_program=True` and `modules_to_serialize` are provided, it will register those modules for serialization
    with cloudpickle's `register_pickle_by_value`. This causes cloudpickle to serialize the module by value rather
    than by reference, ensuring the module is fully preserved along with the saved program. This is useful
    when you have custom modules that need to be serialized alongside your program. If None, then no modules
    will be registered for serialization.

    We also save the dependency versions, so that the loaded model can check if there is a version mismatch on
    critical dependencies or DSPy version.

    Args:
        path (str): Path to the saved state file, which should be a .json or .pkl file when `save_program=False`,
            and a directory when `save_program=True`.
        save_program (bool): If True, save the whole module to a directory via cloudpickle, otherwise only save
            the state.
        modules_to_serialize (list): A list of modules to serialize with cloudpickle's `register_pickle_by_value`.
            If None, then no modules will be registered for serialization.

    """
    metadata = {}
    metadata["dependency_versions"] = get_dependency_versions()
    path = Path(path)

    if save_program:
        if path.suffix:
            raise ValueError(
                f"`path` must point to a directory without a suffix when `save_program=True`, but received: {path}"
            )
        if path.exists() and not path.is_dir():
            raise NotADirectoryError(f"The path '{path}' exists but is not a directory.")

        if not path.exists():
            # Create the directory (and any parent directories)
            path.mkdir(parents=True)
        logger.warning("Loading untrusted .pkl files can run arbitrary code, which may be dangerous. To avoid "
                      'this, prefer saving using json format using module.save("module.json").')
        try:
            with serialize_by_value(modules_to_serialize), open(path / "program.pkl", "wb") as f:
                cloudpickle.dump(self, f)
        except Exception as e:
            raise RuntimeError(
                f"Saving failed with error: {e}. Please remove the non-picklable attributes from your DSPy program, "
                "or consider using state-only saving by setting `save_program=False`."
            )
        with open(path / "metadata.json", "wb") as f:
            f.write(orjson.dumps(metadata, option=orjson.OPT_INDENT_2 | orjson.OPT_APPEND_NEWLINE))

        return

    if path.suffix == ".json":
        state = self.dump_state()
        state["metadata"] = metadata
        try:
            with open(path, "wb") as f:
                f.write(orjson.dumps(state, option=orjson.OPT_INDENT_2 | orjson.OPT_APPEND_NEWLINE))
        except Exception as e:
            raise RuntimeError(
                f"Failed to save state to {path} with error: {e}. Your DSPy program may contain non "
                "json-serializable objects, please consider saving the state in .pkl by using `path` ending "
                "with `.pkl`, or saving the whole program by setting `save_program=True`."
            )
    elif path.suffix == ".pkl":
        logger.warning("Loading untrusted .pkl files can run arbitrary code, which may be dangerous. To avoid "
                      'this, prefer saving using json format using module.save("module.json").')
        state = self.dump_state(json_mode=False)
        state["metadata"] = metadata
        with open(path, "wb") as f:
            cloudpickle.dump(state, f)
    else:
        raise ValueError(f"`path` must end with `.json` or `.pkl` when `save_program=False`, but received: {path}")
set_lm(lm)

Set the language model for all predictors in this module.

This method recursively sets the language model on every leaf module in this module.

Parameters:

Name Type Description Default
lm

The language model instance to use for all predictors.

required

Examples:

>>> import dspy
>>> lm = dspy.LM("openai/gpt-4o-mini")
>>> program = dspy.Predict("question -> answer")
>>> program.set_lm(lm)
Source code in dspy/primitives/module.py
def set_lm(self, lm):
    """Set the language model for all predictors in this module.

    This method recursively sets the language model on every leaf module
    in this module.

    Args:
        lm: The language model instance to use for all predictors.

    Examples:
        >>> import dspy
        >>> lm = dspy.LM("openai/gpt-4o-mini")
        >>> program = dspy.Predict("question -> answer")
        >>> program.set_lm(lm)
    """
    for _, param in self.named_parameters():
        if isinstance(param, Module):
            param.lm = lm