Setting up DSPy¶
Install DSPy with pip install dspy or run uv add dspy to add DSPy to a virtual environment. DSPy works in Python 3.10+ environments.
For the optional LiteLLM compatibility engine, install pip install "dspy[litellm]" or run uv add "dspy[litellm]". Native imports and calls do not require LiteLLM. If a call selects compatibility without the extra installed, DSPy raises an error with the installation command. Application-owned integrations can also use this extra; see the bounded text-only Bedrock Nova recipe.
The extra excludes LiteLLM 1.106.0–1.106.7 and 1.108.0–1.108.3. Those releases import MCP 2-only gateway code during ordinary Responses calls and Chat calls with tools, even when no MCP gateway is requested. LiteLLM 1.108.4 fixes that unnecessary import (upstream fix). Older versions outside this affected interval remain eligible; installing the extra does not require MCP or upgrade it to version 2. The exclusion also applies when MCP 2 would make an affected release work, because package requirements cannot express a constraint conditional on another package’s installed version. For actual LiteLLM MCP gateway use, install LiteLLM’s own mcp extra and follow its version-specific requirements; DSPy’s separate mcp extra does not activate it.
Connecting to a language model¶
DSPy connects to language models with the dspy.LM class. To set up a language model, we provide a "provider/model" format string and an API key:
import dspy
# Pass the key explicitly...
lm = dspy.LM("openai/gpt-5-nano", api_key="YOUR_OPENAI_API_KEY")
dspy.configure(lm=lm)
Behind the scenes, DSPy uses its bundled native lm15 engines to translate a common request/response contract into provider protocols. No separate lm15 installation is needed. engine="auto" (the default) prefers native and may select optional LiteLLM for unsupported routes, settings or representations before provider I/O. engine="lm15" requires native; engine="litellm" explicitly selects compatibility. Authentication failures, timeouts and other provider failures never switch the backend.
You can replace the model and credentials with a supported native provider such as Anthropic or Gemini. Model features and authentication differ by provider; not every historical provider prefix is supported. See other native providers below and custom providers and engines for other backends. If you are upgrading, review the breaking changes.
Once we have an LM, calling dspy.configure(lm=lm) sets our LM as the default provider for every DSPy program in the process. This sets our LM globally, but we can selectively override this with dspy.context when more granular control is needed.
Let’s ensure everything works by manually calling the model:
Which returns:
With an LM configured, we can proceed to writing our first program.
Other native providers¶
Set the appropriate credentials before constructing your LM:
| Service | Example model | Configuration |
|---|---|---|
| Anthropic | anthropic/claude-sonnet-4-5-20250929 | ANTHROPIC_API_KEY or api_key= |
| Google AI Studio | gemini/gemini-2.5-pro | GEMINI_API_KEY or api_key= |
| Azure OpenAI | azure/your-deployment-name | AZURE_OPENAI_API_KEY and AZURE_OPENAI_ENDPOINT (resource root) |
| Vertex AI Gemini | vertex:gemini-2.0-flash | Application Default Credentials, GOOGLE_CLOUD_PROJECT, and GOOGLE_CLOUD_LOCATION |
| Local Ollama | ollama_chat/llama3.2:1b | Start Ollama and pull the model; the native route supplies its local address and placeholder key |
For example, after setting ANTHROPIC_API_KEY:
Native Vertex uses its own vertex: route, not the LiteLLM vertex_ai/ prefix or vertex_project / vertex_location kwargs. A service-account file can be selected with GOOGLE_APPLICATION_CREDENTIALS. Set GOOGLE_CLOUD_LOCATION explicitly when region matters; its default is global. Azure LM calls use the native v1 route, not the old api_version configuration. Use model_type="responses" for a native Responses endpoint; model_type="text" requires optional LiteLLM compatibility.
For an OpenAI-compatible Chat Completions server using bearer authentication:
lm = dspy.LM(
"openai/your-model-name",
api_base="https://your-provider.example/v1",
api_key="YOUR_PROVIDER_KEY",
)
An unlisted service can be registered when it uses a supported wire dialect. Other protocols or authentication flows can use optional LiteLLM compatibility where supported, or a custom engine. Compatibility support does not establish native parity. Embeddings have a separate OpenAI SDK or callable interface, with no LiteLLM fallback.
Next: Your first program →