Skip to content

Flex: Optimizable module code

Intent

dspy.Flex is for tasks where you don’t know the right shape of the solution up front. Every other module fixes its structure when you construct it — Predict is one call, ChainOfThought is one call with reasoning, ReAct is a tool loop — and optimization only tunes the prompt around that fixed structure. Flex moves the structure itself into the search space. You give it a signature; it starts as a dspy.Predict or dspy.RLM baseline; and dspy.GEPA rewrites its entire implementation — how many predictors, which primitives, what runs in Python instead of an LM — against your metric. Reach for it when the best decomposition is something you’d rather discover and optimize.

Design decisions

1. A Flex is a module whose source code is the optimizable parameter

An ordinary module’s tunable surface is its predictors’ instructions. A Flex’s tunable surface is a whole dspy.Module subclass, held as a source string and exposed as module_src. That class has the usual two methods: __init__ constructs whatever predictors it needs, and forward calls them and returns a dspy.Prediction. Optimizing the module means replacing that source with a better version — different predictors, different control flow, more or less Python. The prompt is no longer the unit of optimization; the program code is.

2. It drops into any signature and starts from a simple baseline

You construct Flex from any dspy.Signature, and it’s immediately runnable. With no tools, its baseline source is a single dspy.Predict over the whole signature; with tools, a dspy.RLM so the baseline can call them. The baseline is the simplest thing that works as the starting point of a search.

3. GEPA discovers Flex by type and optimizes code instead of text

When GEPA compiles a program, it enumerates the Flex submodules and splits its work: each Flex becomes a code component, every other predictor stays an instruction component. Code components are seeded with their current module_src and evolved by a dedicated code proposer; instruction components are seeded with their current instructions and evolved by GEPA’s usual instruction proposer. Each side has its own override: a custom instruction_proposer replaces the instruction proposer, a custom code_proposer replaces the code proposer, and whichever you leave unset keeps its default.

4. The code proposer reflects on whole-program behavior

Instruction optimization in GEPA is per-predictor: it looks at one predictor’s inputs, outputs, and feedback. Code optimization can’t work that way, because the predictors are part of what’s being rewritten — they may not exist in the next candidate. So a code component reflects on the whole program’s I/O instead: the module’s inputs, its final prediction, and the metric’s feedback per example. The proposer is given the signature, any available tools, a catalog of allowed primitives, the current source, and a batch of failing examples, and asked to return a full revised module class.

5. Predictors inside a Flex are owned by its code, not tuned in parallel

When you mix a Flex with ordinary modules in one program, GEPA optimizes the Flex’s code and the other predictors’ instructions — but never the instructions of predictors that live inside the Flex. Those predictors are constructed by the current module_src and will be replaced wholesale by the next code candidate, so tuning their instructions would be optimizing something that may be overwritten.

Flex is a subclass of Parameter and Module (like dspy.Predict), so a parent program’s named_parameters() yields it as one leaf and never recurses into it. A parent’s named_predictors(), predictors(), set_lm(), and demo/instruction optimizers like BootstrapFewShot don’t see the Flex’s internals. The Flex reports the same about itself: flex.named_predictors() is empty — its update unit is its code, not the predictors the code constructs. A Flex can carry its own LM (flex.set_lm(...)), which forward applies as the ambient LM for the whole call; otherwise the LM calls its code makes resolve the ambient LM at call time (dspy.configure(lm=...), or a caller’s dspy.context(lm=...)). reset_copy() resets a Flex the way it resets a Predict: the LM is cleared while module_src is kept, just as tuned instructions are.

6. A broken candidate scores as a failure, not a crash

The reflection model authors code, and code can be wrong. A candidate that doesn’t parse raises when GEPA binds it to build the program for evaluation; Flex optimization catches that, logs it, and scores the whole batch at the failure score, letting the search continue and simply not select the broken candidate. Code that parses but breaks when it runs — an import that doesn’t resolve, an edge case that throws on some inputs — fails per example at forward instead: each crashed example is scored at the failure score in its own slot, by example index, so the surviving scores stay aligned to the batch and GEPA’s per-instance bookkeeping stays intact. Either way, the optimization is robust to the reflection model’s mistakes by construction.

7. Generated code always runs through an interpreter

Flex takes its interpreter from interpreter_factory, else from dspy.settings.interpreter_factory, else from dspy.PythonInterpreter (Deno/Pyodide). Each of those must be a zero-argument factory: a bare instance or None is rejected. The generated code imports the real dspy in that interpreter. With the default, everything stays in the sandbox except provided-tool calls and LM calls, which go to the host. The host builds each request from its own LM, admits only generation options, counts the call against max_predictor_calls, and runs it, so the model, credentials, cache, and retries stay on the host. Each code-executing module that the generated code builds, like dspy.RLM, gets a fresh interpreter from the same factory, so its code shares nothing with the module that built it. Functions of that module it is given, like an llm_query or a helper, still run in the module’s sandbox. Custom factories define their own trust boundary: LocalInterpreter separates process memory and stdout but retains the host user’s filesystem, environment, credentials, network, and process authority. An in-process interpreter (one that runs code with exec() in the host, such as dspy_interpreters.InProcessInterpreter) works but isolates nothing: generated code uses the host’s own dspy module, so max_predictor_calls and the LM-option allowlist bind only well-behaved code. Use one only for code you trust. Any other interpreter must let the generated code import the host’s dspy. The factory creates a fresh interpreter for each forward, and one for each code-executing module the generated code builds, all shut down when the forward ends. Source portability between custom interpreters is not guaranteed.

8. The declared output types are enforced at the sandbox boundary

Everything the generated code returns crosses back from the sandbox, so Flex parses each declared output field against its annotation on the way out, and each input on the way in. A Flex returns the same types as a dspy.Predict over the same signature and stays substitutable for one. The custom types of the Flex’s signature have to exist in the sandbox too, so the generated code can name them (text: str -> person: Person, or person: Person = dspy.OutputField() in a class). dspy’s own types, the standard library’s and those of the packages dspy uses are imported there. Your own types stay yours: the host owns them, and no code of yours crosses. In the sandbox, a value of one of your types is plain data: the JSON your type writes for a round trip, like a dict of a model’s fields, without any field marked exclude=True. The generated code reads it that way, and returns such data where the signature names the type, which the host reads back as your real type, so your validators and defaults apply there. The code can name the type, as in a signature, but can’t build a value of it, since the type and its code stay on the host. Methods and properties stay on the host, as does the formatting and parsing of every predictor call: predictors in the sandbox send the host’s adapter the signature, demos and inputs, and it formats them with your real types, calls the metered LM and parses the answer there, as without a sandbox. Other values carry their types across, so tuples, sets, bytes, datetimes and dspy’s own types arrive as themselves. A type the generated code defines itself reaches the host as plain data, and the host formats it as text, which is parsed into that type in the sandbox: the host never builds a class from what the sandbox sends. And a candidate whose output doesn’t match — wrong shape, or a declared field missing entirely — raises a CodeInterpreterError naming the field, which GEPA scores at the failure score for that example rather than handing the metric a Prediction that breaks somewhere else. An annotation that can’t round-trip through a signature string at all (a Callable, say) is emitted untyped rather than rendered into source that wouldn’t bind.

9. The code is state; the interpreter is a runtime dependency

A saved Flex is {"module_src": ..., "lm": ...} — the code, plus any LM set directly on the module. The internal predictors are not saved: they are derived from the code, and each forward reconstructs them from the bound source. The interpreter is a live runtime resource and is not serialized; save(path, save_program=True) cloudpickles the program with the bridge excluded, and loading rebuilds it. Reconstructing with dspy.Flex(signature) restores the configured sandbox, so you only re-supply the interpreter_factory before load if you optimized with one passed to the constructor.

API walkthrough

Defining and running a Flex

dspy.Flex(signature, *, tools=None, interpreter_factory=PythonInterpreter, max_predictor_calls=100) Parses the signature and binds the baseline source — a single dspy.Predict over the signature, or a dspy.RLM when tools are given. Validates interpreter_factory as a zero-arg factory and sets up the sandbox bridge. dspy.configure(interpreter_factory=...) replaces the default. A factory passed to Flex wins, unless it is PythonInterpreter.

__call__(**inputs) / forward(**inputs) Runs the currently bound source inside the interpreter, where every LM call goes to the host. Returns a dspy.Prediction over the signature’s output fields, and accepts keyword inputs only.

module_src A read-only property holding the current implementation as source — one dspy.Module subclass. This is the value GEPA reads as the seed and overwrites with each accepted candidate.

Optimizing with GEPA

dspy.GEPA(metric=..., reflection_lm=..., ...).compile(flex_program, trainset=..., valset=...) Compiling a program that contains one or more Flex submodules optimizes each Flex’s code and every non-flex predictor’s instructions together, under one budget. Returns a new program whose module_src (per flex submodule) is the best code found. The auto-budget counts each flex submodule as one component alongside the instruction predictors.

The metric’s feedback As with any GEPA metric, the feedback string goes into the prompt handed to the proposer — here, the code proposer. Feedback that diagnoses why an output was wrong and hints at structure steers rewrites far better than a bare score.

Trace-aware metrics (program_trace) Add program_trace=None as a sixth parameter to your metric and GEPA passes the execution trace at scoring time, so you can score against how the answer was produced. For instance, you may use len(program_trace) as an LM-call count to fold a small per-call penalty into the score.

Tools and sandboxing

tools=[...] Plain functions or dspy.Tool instances, referenced by name in the generated code, so each name must be a valid Python identifier. Providing tools makes the baseline a dspy.RLM and tells the code proposer they are in scope — it can wire them into dspy.RLM/dspy.ReAct, call them directly, or supplement them with its own inline helpers.

interpreter_factory=... Defaults to dspy.PythonInterpreter (sandboxed, needs Deno). dspy.configure(interpreter_factory=...) replaces that default on each interpreter session, including for a Flex built before the call. Must be a zero-argument callable returning a fresh CodeInterpreter for each interpreter session, so parallel evaluations receive isolated sessions. Any interpreter other than dspy.PythonInterpreter must let the generated code import the host’s dspy. As in dspy.RLM, a bare interpreter instance is not accepted. This low-level hook does not guarantee source, standard-library portability, or a particular security boundary between different interpreters.

max_predictor_calls The maximum number of LM calls the generated code can make in one forward, counted by the host as each call arrives. Every predictor call makes at least one. It guards against runaway loops. None removes the limit.

What the generated code can use

The optimizer-authored code runs against the real dspy package, the same source the host runs. Only a few things cross to the host:

  • LM calls. Every predictor uses the host’s LM through a stand-in that sends each request to the host, and each predictor call is formatted and parsed there by the host’s adapter. Generation parameters (temperature, max_tokens, n, …) apply, while LM routing and credential options (api_base, api_key, headers, model) are rejected by the host. The host runs each call on its LM’s sync path, an async one in the sandbox too, since the caller’s loop waits while the sandbox runs. An LM that only implements async can’t serve a sandbox.
  • Provided tools. Each tool passed to dspy.Flex(tools=...) is in scope by name as a dspy.Tool carrying its host name, description, and arguments, so the code can call it directly or pass it to dspy.ReAct or dspy.RLM. It runs on the host. Functions the code defines run in the sandbox and can be passed to those modules as well.
  • Custom types of the signature, as described in section 8.
  • The trace. Predictor calls made in the sandbox are added to the host’s dspy.settings.trace, which is how program_trace metrics see them.

The code should build its modules from the primitives the proposer is told about (dspy.Predict, dspy.ChainOfThought, dspy.ReAct, dspy.RLM, string or class-based dspy.Signatures, dspy.Tool, dspy.Prediction) and the Python standard library, imported inside forward. It should not construct LMs or change settings, since the host owns the LM. dspy.PythonInterpreter has no filesystem, network, or threads, so dspy.Parallel and batched calls run one after another there. Anything that fails when the candidate runs is scored at the failure score, so a mistake costs a search step rather than crashing the run. The proposer-facing version of this list is dspy/predict/flex/primitives_doc.py.

The sandbox bundle The first time a machine runs a Flex forward (or a dspy.RLM call) on dspy.PythonInterpreter, the host reads dspy’s dependencies from its installed metadata, takes Pyodide’s builds of those Pyodide ships (from the Pyodide CDN, checked against Pyodide’s lock file) and this environment’s copies of the other pure-Python ones, and byte-compiles them together with dspy’s own source into $DSPY_CACHEDIR/sandbox (~/.dspy_cache/sandbox by default). It also records the memory of a sandbox that has imported dspy. That takes a few seconds, once, and again whenever the dspy source changes. After that, each sandbox restores that snapshot, so it starts with dspy already imported in about a quarter of a second.

Saving and loading

save(path) / load(path) / dump_state() / load_state(state) module_src travels in the serialized state (along with any LM set on the module), so loading restores the optimized code; the predictors are rebuilt from it on each forward. The interpreter is not serialized; reconstructing with dspy.Flex(signature) restores the configured sandbox, so re-supply it in the constructor before load only if you passed a custom interpreter_factory there.